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ABSTEACT 

The  appropriateness  of  a  multidimensional  scaling  tech- 
nique (HDS)  in  attempting  to  Quantify  students  perceptions 
of  teacher  performance  is  investigated  in  the  following 
study.  Data  collected  on  an  interactive  computer  survey  and 
from  Student  Opinion  Forms  (SOF*s)  are  used  to  determine  if 
satisfactory  linear  relationships  for  teacher  performance 
exist-  Multiple  linear  regression  and  factor  analysis 
attempt  to  identify  what  appear  to  be  the  most  important 
characteristics  in  instructor  performance  according  to  the 
perceptions  of  a  control  group.     Spatial  plots  are  created 
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I.  INTEODDCTION 

A.   PURPOSE 

The  purpose  of  this  thesis  is  twofold: 

1.  To  develop  methodology  that  will  help  discover  the 
important  characteristics  of  instructor  performance 
as  perceived  by  each  student  group; 

2,  To  develop  user  friendly  software  compatible  with  our 
IBM  3033  system  that  facilitates  the  data  collection 
and  processing  in  support  of  the  foregoing. 

The  methods  designed  herein  request  proximity  data  or 
similarity/dissimilarity  data  on  pairwise  combinations  of 
professors  in  the  Operations  Research  department.  Also,  the 
respondents  are  requested  to  provide  ratings  on  several 
'bipolar  scales'  of  suggested  instructor  characteristics. 
The  methods  for  discovering  the  dimensions  or  relationships 
that  appear  to  characterize  the  professors  utilize  several 
statistical  tools  including  multidimensional  scaling, 
regression  analysis,  factor  analysis  and  cluster  analysis. 
The  data  used  in  the  analysis  comes  from  those  students  in 
the  Operations  Research  curriculum  graduating  in  March  of 
1985. 

An  interactive  computer  survey  is  designed  to  query  the 
students  on  their  perceptions  of  the  teaching  effectiveness 
of  the  instructors.  Thus,  data  is  input  by  the  students 
during  a  twenty  minute  session  on  the  3273  terminal. 
Initially  the  student  links  to  the  software,  and  when 
finished,  transmits  his  responses  to  a  central  file. 


B.  SUBJECT 

The  suhject  of  this  thesis  deals  with  quantifying 
perceptions.  Specifically,  we  woald  like  to  obtain  an 
explanation  as  to  why  students  perceive  instructors  as  being 
similar  or  dissimilar  and  discover  the  dynamic  factors  that 
a  particular  class  uses  to  discriminate  among  instructors. 
The  multidimensional  scaling  technique  uses  the  information 
from  the  survey  to  create  a  'multidimensional  map*  of 
points.  Each  point  represents  the  coordinate  position  of 
the  objects  under  investigation,  in  our  case,  professors. 
Once  this  spatial  plot  is  produced,  it  remains  for  the 
researcher  to  discover  those  factors  that  appear  to  cause 
the  structural  relationships. 

Multiple  regression  analysis  and  factor  analysis  are  two 
techniques  commonly  used  to  describe  linear  relationships 
among  dependent  and  independent  variables.  Each  method  is 
given  consideration  here  in  attempting  to  interpret  the 
spatial  plot  produced  by  the  multidimensional  scaling 
program,  KYST  [Ref.  1].  Additionally,  cluster  analysis  is 
used  to  group  the  professors  into  disjoint  clusters.  This 
cluster  information  is  presented  to  the  students  during  exit 
interviews  to  help  guide  the  researcher  in  his  attempt  to 
find  the  underlying  relationships. 

C.  SCOPE 

This  thesis  is  presented  in  four  major  chapters 
excluding  the  introduction.  The  second  chapter  describes  in 
detail  the  background  of  multidimensional  scaling  methods. 
A  brief  account  of  regression  analysis,  factor  analysis  and 
cluster  analysis  is  given  as  well.  The  third  chapter  speci- 
fies the  means  by  which  the  data  was  collected  in  the  inter- 
active survey.  Chapter  IV  emphasizes  the  analysis  of  the 
data.  Finally,  Chapter  V  provides  a  summary  of  the  salient 
points   determined    in    the   study. 


D.   A  BBIEF  SOaHABI 

A  four  dimensicnal  interpretation  was  emphasized  in 
describing  the  data  obtained  from  the  computer  survey  and 
the  SOF  forms.  These  four  factors  included,  1)  a  student- 
instructor  interaction  effect;  2)  the  degree  to  which  a 
professor  was  perceived  as  being  organized  or  prepared  for 
class;  3)  a  combined  effect  of  grading  policy,  effort 
required  outside  class  and  pace  of  the  course;  and  U)  a 
composite  effect  combining  class  size  and  the  degree  to 
which  a  course  relied  upon  pr erei^uisites.  A  high  correla- 
tion appears  to  exist  among  those  bipolar  scales  used  in  the 
current  SOF  form.  Further  investigation  hopefully  will  lead 
to  discovering  other  factors  that  will  help  describe  teacher 
performance.  The  results  obtained  in  this  study  are  not 
meant  to  be  predictive  but  ejcplana tory.  The  value  associ- 
ated with  an  instructor  for  each  characteristic  may  be 
regarded  as  his  score  on  that  dimension.  Thus,  rankings  of 
instructors  by  characteristics  are  possible. 


II.  BACKGROgND 

A.   WHAT  IS  aOLTIDIMEHSIONAL  SCALING? 

Multidimensional  scaling  involves  the  problem  of 
depicting  n  points  in  multidimensional  space  such  that  the 
interpoint  distances  correspond  in  some  manner  to  measured 
proximity  data  [Eef.  2:  p.  1]-  The  proximity  data  can  be 
similarities,  dissimilarities,  correlation  coefficients  or 
any  other  measure  of  association  as  perceived  by  a  set  of 
judges  participating  in  an  experiment.  Multidimensional 
scaling  techniques  attempt  to  produce  the  structure  or 
interrelationships  among  the  n  objects  by  assuming  a  direct 
correspondence  between  the  measured  proximity  data,  or 
dissimilarity  data  S\'  i  in  our  case,  and  the  interpoint 
distances  d-,.*  .  Several  choices  of  multidimensional  scaling 
exist,  the  difference  being  the  assumed  relationship  between 
^.]  and  d^.'  .  The  ultimate  product  of  multidimensional 
scaling  (MDS)  will  be  a  spatial  map  that  displays  the  asso- 
ciation between  the  n  objects  under  investigation.  MDS  has 
found  considerable  application  in  the  social  sciences, 
particularly  in  the  realms  of  psychology,  sociology, 
economics  and  education. 

A  significant  point  worth  pursuing  is  this  idea  of 
correspondence  between  the  proximity  data  ;f.,  ,  and  the 
distance  data  d-  .  A  fairly  simple  method  of  analyzing  a 
possible  relationship  would  be  to  observe  a  scatter  plot 
similar  to  the  one  given  in  Figure  2.  1  .  The  vertical  Y 
axis  of  the  scatter  plot  contains  the  measured  dissimilari- 
ties i.'j,  while  the  horizontal  X  axis  shows  the  corresponding 
distances,  d;-  ,  computed  from  the  derived  set  of  character- 
istic vectors. 
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Figure  2.1    Scatter  Plot  of  Distance  vs.  Dissiailarity  Data 
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It  can  be  seen  that  an  ascending  pattern  is  created  between 
the      distances    and      the      dissimilarities,  Ona    might      even 

suggest  that  the  relationship  is  linear,  and  could  be 
described        by         an        equation        of  the        form        a+         bx. 

Multidimensional  scaling  methods  that  use  a  formula  to 
describe  distance  as  a  function  of  dissimilarities  are  known 
as  metric  HDS.  Metric  MDS  uses  the  numerical  properties  of 
the  proximity  data  to  determine  distances.  Another  means  by 
which  distances  are  created  from  dissimilarity  data  without 
using  the  numerical  properties  of  the  dissimilarity  data  is 
known  as  nonmetric  UBS.  Nonmetric  MDS  relies  totally  upon 
the  rank  ordering  of  the  dissimilarities  to  produce  the 
distance  data  [Ref-  3:  p-  22]-  One  would  normally  expect 
small  dissimilarities  to  correspond  to  small  distances  and 
large  dissimilarities  to  correspond  to  large  distances. 
Although  this  relationship  is  rarely  a  perfect  one,  the  rank 
orderings  of  the  dissimilarity  data  is  usually  enough  to 
create  a  good  fit.  Shepard  and  Kruskal  [Ref.  2:  p.  2],  have 
done  a  substantial  amount  of  work  in  nonmetric  MDS  and  this 
particular    method   will  be   followed   in   this    study. 

A  review  of  some  notation  might  be  appropriate  at  this 
point.  The  n  objects  about  which  the  investigator  is  trying 
to  ascertain  some  fundamental  relationship,  can  really  be 
any  set  of  stimuli.  For  example,  one  might  be  interested  in 
discerning  the  perceived  distances  between  political  candi- 
dates with  hopes  of  discovering  what  issues  or  dimensions 
really  set  them  apart  from  each  other  in  the  minds  of  the 
judges.  Or,  one  might  be  interested  in  discovering  the 
perceived  distances  between  countries  in  order  to  explain 
how  some  countries  might  react  in  some  political  or  economic 
situations.  For      this      study,         the      n      objects      are      the 

professors  in  the  Operations  Research  department  and  wc  are 
interested  in  determining  what  factors  cause  the  professors 
to    be   similar    or      dissimilar   in   the    minds    of    a      fixed    set    of 
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judges.  Each  professor  can  be  thought  of  as  a  point  in 
multidimensional  space.  If  it  takes  t  dimensions  to  accu- 
rately describe  this  multidimensional  space,  then  the  coor- 
dinate describing  professor  x   would  be 

^i~v^ii  r"*"X;s  '•••^ife)  • 

For  n  professors  the  coordinate  system  that  results  looks 
like 


x,=  (x,,  ,...x^^ 


^J 


f   *    m   •X. 


ns 


.  .X 


it. 


) 


Thus,   the  entire  space  contains  a  configuration  of  n  points 
each  of  t  components. 

Each  judge  or  student  is  asked  to  complete  a  survey  that 
requires  him  to  provide  a  value  corresponding  to  how  similar 
or  dissimilar  he  perceives  each  pairwise  combination  of 
professors  to  be.  The  scale  used  in  the  survey  associates 
the  value  1  with  the  meaning  very  similar  and  the  value  of  9 
vith  the  meaning  very  dissimilar.  In  order  to  determine 
rather  than  impose  the  relationships  between  professors,  the 
characteristics  on  which  the  professors  are  scored  are  not 
specified  at  this  time  [Ref.  3:  p.  9].  The  notation  for 
dissimilarity  data  in  this  study  is  (Tl,  .  This  value  repre- 
sents the  perceived  dissimilarity  between  professor  i  and 
professor  j.  The  end  result  of  the  data  collection  is  a 
matrix  of  dissimilarity  values  for  n  stimuli  that  looks  like 
the  following. 


•i,, 


K. 


L  -i., 


S.^i 
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In  this  study  the  dissimilarity  values  ^r  and  iy^  were 
assumed  to  be  the  same  (symmetric  matrix)  ,  and  the  values  6, 
f  were  ignored.  This  results  in  a  partial  matrix  with  upper 
triangular  or  lower  triangular  form  not  including  the  main 
diagonal.  The  total  possible  pairwise  comparisons  are  then 
n(n-1)/2. 

The  actual  distance  between  professors  i  and  j,  denoted 
d'-  ,  is  calculated  as  a  euclidian  distance  in  the  following 
manner  for  t  dimensions 

like  the  similarity/dissimilarity  matrix,  the  end  result  is 
a  matrix  of  distances  such  that  d,-  =d-  and  d-  =0.  We  point 
out  here  that  although  we  have  specified  the  euclidean 
distance  as  the  method  by  which  distances  are  computed  in 
the  computer  algorithm,  it  is  possible  to  substitute  non- 
euclidean  distances  of  the  form 


d    (x,y)  = 


rix,-y..  r 


Vr 


for   r> 1 


In  the  field  of  mathematics,  these  distances  are  known  as 
Minkowski  r-metrics  and  are  true  distances  in  the  sense  that 
they    satisfy   the    triangle   inequality 

d^  {x,z)<    d^  (x,y)+dj.  (y,z)  . 

Euclidean  distances  and  Minkowski  r-metrics  share  many  prop- 
erties, however,  they  do  differ  when  it  comes  to  rotating 
the  solution-  When  rotations  are  involved,  any  rigid  rota- 
tion leaves  euclidean  distances  unchanged.  The  only  rigid 
rotations  that  leave  non-euclidean  distances  unchanged  are 
rotations  that  transform  all  permutations  of  coordinate  axes 
into   coordinate    axes    [Eef-    2:    pp,    22-23]. 

Tinally,  the  number  of  dimensions  used  to  output  the 
final  mapping  is  not  restricted  in  any  way  mathematically 
other   than   NT<N(N-1)/2.         However,    any   visual   interpretation 
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in  greater  than  three  dimensions  becomes  extremely  difficult 
for  anyone  trying  to  discover  the  meaning  of  the  grouping's. 
The  stress,  or  goodness-of-f it  measure,  which  is  described 
in  the  next  section,  is  usually  expected  to  reveal  the  test 
dimension   for   analysis. 

B.       GOODMESS    OF    FIT: THE    STRESS 

A  performance  indicator  for  each  choice  of  t  is  needed. 
The  customarily  used  function  is  known  as  the  stress. 
Stress  incorporates  a  fitting  technigue  that  measures  the 
degree  of  nonmonotonicity  between  the  dissimilarities  i'.,  , 
and  the  distances  d_  .  If  a  configuration  of  points 
existed  such  that  a  perfect  monotone  relationship  prevailed 
between  the  dissimilarity  data  and  the  distance  data,  then  a 
perfect  match  would  occur  and  the  stress  would  be  zero  for 
that  particular  dimension  size  and  all  dimensions  greater. 
Unfortunately,  zero  stress  rarely  if  ever  occurs  naturally 
in  data  samples.  The  best  choice  then  is  to  determine  that 
configuration  of  points  which  minimizes  the  stress  for  each 
choice  of  dimension,  t.  The  method  used  to  determine  stress 
uses  least  squares  monotonic  regression,  suitably  normalized 
to  produce  a  non-dimensional  value  that  indicates  goodness 
of  fit  [Bef.  2:  pp.  2-3]-  The  lower  the  value  of  stress, 
the  tetter  the  fit.  Kruskal  [Ref.  2:  p.  3],  has  been  able 
to  associate  a  verbal  description  with  some  specific  values 
of  stress  based  upon  his  experience.  These  values  are  indi- 
cated in  Table  1  -  In  almost  every  instance,  by  increasing 
the  number  of  dimensions  that  describe  the  data,  the  value 
of  the  stress  decreases.  However,  one  generally  anticipates 
that  the  amount  of  benefit  associated  with  incrementing  the 
dimensionality  is  marginally  insignificant-  In  other  words, 
a  tradeoff  exists  between  keeping  the  output  in  a  lower 
dimension  to  make  interpretation  easier,  if  the  stress 
improves   only    slightly. 
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TABLE    1 
Table  of    Stress   Values   According   to   SrusJcal 


STRESS  VALUE 
.20 
.  1  0 
.05 
.  025 
0  .0 


GJOONfc  S5  OF  Fir 

FAIR 
GGOD 
EXCELLENT 
PEwFECr 


Figure  2-2  illustrates  an  example  of  what  we  mean  by  a 
non-perfect  match  between  the  dissimilarity  and  the  distance 
data.  Here,  the  deviations  measured  along  the  horizontal 
distance  axis  ,d-,,  ,  between  the  starred  coordinates  and  the 
circular  coordinates,  indicate  the  degree  to  which  this 
configuration  does  not  meet  the  monotonicity  requirement. 
The  values  d:,  are  defined  to  be  those  numbers  measured  from 
the  horizontal  X  axis  that  minimize  stress  subject  to  the 
constraint  of  monotonicity  [Ref-  2:  pp.  8-9]. 

C.   DIHEHSIONAL  BEQOIBEHENTS 

The  choice  of  how  many  dimensions  are  reguired  to 
completely  specify  the  output  from  the  multidimensional 
scaling  program  is  certainly  not  intuitively  obvious.   There 
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are  metiiods,  however,  which  help  to  indicate  why  one  choice 
of  dimensionality  would  be  more  appropriate  than  others. 
The  most  obvious  of  these  is  to  compare  how  stress,  the 
goodness-of-fit  measure,  improves  as  a  function  of  dimen- 
sion. One  way  to  do  this  is  to  compute  the  best  configura- 
tion for  several  dimensions  and  create  a  plot  of  stress  vs. 
dimension  to  visually  compare  the  results.  It  should  be 
pointed  out  that  the  more  dimensions  one  uses  to  explain  or 
interpret  the  data,  the  lower  the  stress  becomes.  When  the 
number  of  dimensions,  t,  exceeds  the  number  of  objects  minus 
one,  n-1,  the  stress  will  always  be  zero  [Eef.  2:  p.  16]. 
what  one  wants  to  look  for  is  that  dimension  above  which  the 
stress  improves  only  slightly.  If  the  data  is  good,  a 
noticeable  elbow  will  show  up  in  the  plot  to  indicate  the 
appropriate  dimension.  He  were  not  so  fortunate  in  our 
present  study.  Figure  2.3  illustrates  a  stress  versus 
dimension  plot. 

Probably  more  than  anything  else,  interpretability 
should  be  considered  a  key  criterion  to  use  in  selecting  the 
appropriate  dimension  for  analysis-  If  it  is  possible  to 
interpret  the  results  of  an  MDS  configuration  in  two  dimen- 
sions more  readily  than  in  say  three  dimensions,  even  though 
the  stress  is  lower  in  three  dimensions,  one  should  consider 
employing  the  two  dimensional  interpretation.  A  final 
criterion  suggested  by  Kruskal  [Eef.  2:  p.  16],  depends  upon 
the  accuracy  of  the  data.  If  an  independent  estimate  exists 
to  corroborate  error  free  or  near  error  free  data,  then  one 
is  allowed  to  extract  more  dimensions  than  one  would  under 
more  error  prone  conditions.  'Rhen  all  is  said  and  done,  the 
choice  of  dimensionality  rests  largely  upon  the  experience 
of  the  experimenter. 
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Figure    2-3        Plot  of    Stress    vs.    Dimension 
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D.   THE  MULTIDiaENSIOMAL  SCALING  ALGORITHM 

Since   the  mathematical  technique   for  determining   the 

optimal  configuration,    and  therefore   optimal  stress,    is 

somewhat  complicated,   only  a  brief  description  of   what  is 

considered  necessary  will  be  described  here.   Suppose  that  t 

dimensions  are  selected  to  describe  a  configuration  of  n 
points.   Then 

(X,,     ,...Z;^   ,...X-|;   ,...X,^^j 

can  be  used  to  describe  a  particular  configuration  in  multi- 
dimensional space.  For  this  particular  configuration,  a 
specific  value  of  stress  exists.  The  overall  objective  is 
to  make  the  stress  value  as  small  as  possible.  This  turns 
out  to  be  a  minimization  problem  of  multiple  variables  and 
is  handled  by  the  method  of  steepest  descent.  Specifically, 
the  algorithm  begins  at  an  arbitrary  configuration  and 
attempts  to  improve  itself  by  moving  in  the  direction  that 
improves  or  minimizes  the  stress  value  quickest.  The  direc- 
tion of  movement  is  known  as  the  negative  gradient  and  can 
be  evaluated  from  the  partial  derivarives  of  the  function 

S=f  (Xi,  ,x,i  ,...x^^) 

[Ref.  4:  p.  118].   For  example, 

(-ds/  X,,  ,...  ,-^S/  x^^  ,.  ..,-^S/  X,.  ) 

is  the  negative  gradient.  Once  the  configuration  reaches 
the  point  at  which  it  can  no  longer  improve  in  a  particular 
direction,  the  new  negative  gradient  is  calculated  and  the 
process  continues.  Finally,  when  a  configuration  can  no 
longer  proceed  in  any  direction  with  improvement,  it  has 
reached  a  local  minimum.  Hopefully,  the  local  minimum  is 
also  the  global  minimum,  but  this  is  not  necessarily  true 
all  the  time. 
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It  is  not  always  possible  to  determine  if  the  local 
minimum  is  also  the  global  minimum,  but  some  techniques 
exist  to  help  verify  that  this  is  so.  For  example,  starting 
the  minimization  process  from  several  different  initial 
configurations  and  comparing  the  final  solutions  will  indi- 
cate if  the  same  local  minimum  is  achieved.  The  final 
configuration  with  the  best  stress  value  is  most  likely  the 
global  minimum.  There  is  nothing  to  guarantee  that  one  will 
always  achieve  the  global  minimum,  but  this  is  a  common 
problem  typical  of  non-linear  optimization  problems.  In  any 
case,  the  configuration  is  only  useful  if  in  the  end  it 
makes  sense  and  gives  insight  to  the  experimenter  [Bef.  4: 
p.  119]. 

E.   HOITIPLE  LIHEAR  BEGBESSION 

Once  the  mul tid imesional  scaling  algorithm  computes  the 
configuration  with  the  lowest  stress,  the  researcher  would 
like  to  determine  the  specific  dimensions  that  underlie  the 
data  structure.  One  way  to  do  this  is  to  assume  that  a 
linear  relationship  exists  between  a  dependent  variable  and 
several  independent  variables.  Many  linear  regression 
models  assume  that  the  proportion  of  explained  variation  of 
the  dependent  variable  is  the  sum  of  additive  effects  of 
statistically  significant  independent  variables  [fief,  5:  p. 
54].  Other  regression  models  allow  for  interactive  effects 
between  independent  variables.  The  dependent  variable  is 
said  to  be  regressed  over  the  independent  variables.  The 
result  of  this  regression  process  is  the  coefficient  of 
multiple  determination,  R^  .  'v^^q  value  of  R2  indicates  the 
amount  of  variation  in  the  dependent  variable  explained  by 
the  independent  variables.  The  value  of  R2  ranges  between 
0  and  1-  The  higher  the  value  of  R2  ,i.e,,  the  closer  it  is 
to   1,    the   better   the  model   explains   or   predicts   the 
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dependent  variable.  The  closer  the  value  of  R2  is  to  0,  the 
more  likely  it  is  that  the  model  is  inappropriate  in 
accounting  for  the  variation  in  the  dependent  variable.  The 
reasons  for  a  low  value  of  R2  are  several.  For  one  thing, 
the  relationship  under  investigation  may  not  be  a  linear 
one.  If  this  is  the  case,  linear  regression  modelling  is  no 
longer  a  satisfactory  method  to  use  to  describe  the  rela- 
tionship. However,  given  that  the  relationship  is  truly  a 
linear  relationship,  the  reason  for  a  low  value  of  R2  could 
be  the  result  of  specification  error.  Bhat  is  meant  by  this 
is  that  the  dependent  variable  for  which  an  explanation  is 
sought,  is  being  explained  by  an  inappropriate  set  or  an 
insufficient  number  of  independent  variables. 

The  artwork  in  regression  modelling  as  well  as  in  alter- 
native statistical  modelling  methods  comes  from  being  able 
to  suggest  (or  divine)  the  correct  explanatory  variables. 
In  this  particular  study,  the  explanatory  variables  used 
came  from  current  student  opinion  forms  and  suggestions  from 
previous  students  in  the  Operations  Research  curriculum. 

F.   FACTOR  ANALYSIS 

As  mentioned  before,  factor  analysis  attempts  to  repre- 
sent a  set  of  observed  variables  in  terms  of  a  smaller  set 
of  hypothetical  variables.  The  hypothetical  variables  are 
chosen  to  account  for  the  covariation  among  the  originally 
observed  variables.  The  number  of  common  factors  present 
among  the  observed  variables  can  be  estimated  from  the  rank 
of  the  adjusted  correlation  matrix. 

Difficulties  in  factor  analysis  arise  when  the  factor 
loadings  are  not  known  and  have  to  be  estimated  from  the 
covariance  or  correlation  matrix.  The  problem  is  that  given 
the  correlation  matrix  for  the  observed  variables  is  known, 
any  one   of  many   causal  structures   could  have   produced  it 
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[Ref.  6:  p.  38].  The  three  types  of  problems  that  resiilt 
from  uncertainties  about  the  covariance  matrix  and  the 
underlying  causal  structure  are;  1)  specific  covariance 
matrices  can  be  created  by  factor  models  with  the  same 
number  of  common  factors  but  with  different  factor  loadings; 
2)  specific  covariance  matrices  can  be  created  by  factor 
models  with  different  number  of  common  factors;  3)  certain 
covariance  matrices  can  be  created  by  factor  analytic  causal 
models  as  well  as  non-factor  analytic  causal  models  [fief.  6: 
p.  3  8]- 

Two  assumptions  commonly  made  in  factor  analysis  are  the 
postulate  of  factorial  causation  and  the  postulate  of  parsi- 
mony. Basically,  the  postulate  of  factorial  causation 
requires  that  the  researcher  show  that  the  originally 
observed  variables  are  a  linear  combination  of  some  causal 
variables  [Ref.  6:  p.  43].  The  postulate  of  parsimony 
allows  the  researcher  to  assume  a  factor  model  with  a 
smaller  number  of  factors,  given  that  two  factor  models  with 
different  numbers  of  factors  have  the  same  covariance 
structure  [Ref.  6:  p.  44]. 

G.   CLUSTER  AHALYSIS 

A  final  topic  used  to  assist  in  this  study  is  cluster 
analysis.  Cluster  analysis  is  a  class  of  techniques  that 
typically  place  objects  into  groups  or  clusters  suggested  by 
the  data  such  that  objects  in  a  cluster  tend  to  be  similar 
to  objects  in  the  same  cluster  and  dissimilar  to  objects  in 
other  clusters  [Bef.  7].  The  type  of  cluster  analysis 
followed  in  this  study  is  disjoint  cluster  analysis  whereby 
objects  may  belong  to  one  and  only  one  cluster  as  opposed  to 
hierarchical  cluster  analysis  where  one  cluster  might  be 
contained  within  another- 
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III.  METHOpOLOGI 

A.   LAYIHG  THE  FOONDATION 

The  first  step  in  this  study  to  understand  perceptions 
vas  to  decide  upon  a  control  group  of  students  from  whom 
data  would  be  obtained.  In  this  particular  instance,  the 
judges  were  twenty-three  Operations  Research  students  from 
the  section  that  graduates  in  March  of  19  85.  The  twenty- 
three  students  included  two  foreign  nationals,  two  Marine 
Corps  officers,  four  women  naval  officers  and  fifteen  male 
naval  officers  all  with  various  educational  and  career 
back  grounds. 

The  information  that  was  to  be  gleaned  from  these 
students  was  simply  this:  How  do  professors  in  the 
Operations  Research  department  at  the  Naval  Postgraduate 
School  differ,  and  in  what  ways  are  they  similar  in  teaching 
styles  and  methods?  There  were  several  methods  available  to 
obtain  this  information.  Most  simply,  a  hand  written  survey 
with  many  questions  could  have  been  developed  and  handed  to 
each  student  to  complete  and  return.  The  decision  was  made 
to  handle  the  data  collection  via  a  computer  interactive 
survey  in  order  to  minimize  the  logistics  and  data  manipula- 
tion problems  associated  with  a  hand  written  survey.  The 
logistics  problem  was  not  a  great  problem  per  se,  it  just 
meant  that  a  hand  written  survey  had  to  be  distributed  and 
collected  with  plenty  of  opportunity  for  the  surveys  to 
become  misplaced.  From  a  data  manipulation  point  of  view, 
the  data  collected  from  a  hand  written  survey  would  almost 
certainly  have  to  be  recorded  on  computer  in  order  to  be 
able  to  use  the  results  easily  and  quickly.  As  a  result, 
the  decision  was  made  to   attempt  to  collect  the  information 
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about  the  professors  by  having  the  students  complete  an 
interactive  computer  survey  at  a  Naval  Postgraduate  School 
computer  terminal  with  display  screen.  Although  the  data 
collection  in  this  manner  was  not  entirely  free  from  its  own 
brand  of  difficulties,  it  eliminated  the  requirement  for  any 
paperwork  to  be  handed  out  or  returned.  All  the  results 
were  automatically  sent  to  the  researchers  computer  storage 
at  the  completion  of  the  survey.  Also,  the  data  was  in  a 
ready-to-use  state  by  being  contained  on  computer  disk,  and 
hence  the  need  to  transcribe  results  disappeared. 

Unfortunately,  there  are  numerous  contingencies  for 
which  a  computer  programmer  must  plan  in  order  to  design  a 
computer  survey  that  is  simple,  thorough,  and  user-friendly. 
The  interactive  computer  survey  designed  for  this  study 
served  its  purpose  well.  An  example  of  this  program, 
written  in  fortran,  is  included  in  Appendix  D. 

B.   THE  COMPUTER  SOSVEI 

Several  important  requirements  that  had  to  be  addressed 
in  designing  the  interactive  survey  included  simplicity, 
thoroughness  and  brevity.  It  was  considered  important  to 
collect  accurate  responses  and  to  limit  the  workload  as  auch 
as  possible.  Each  judge  responded  to  one  hundred-seventy 
items  on  the  average.  Even  with  this  many  questions  to 
answer,  each  student  appeared  to  have  completed  the  survey 
on  the  order  of  twenty  minutes-  Even  still,  more  work 
should  be  done  to  reduce  the  amount  of  information  asked  of 
each  judge. 

The  nature  of  the  computer  survey  progressed  as  follows. 
Each  judge  was  required  to  link  to  the  researchers  computer 
disk,  and  once  accessed,  issue  a  command  to  run  the  execu- 
tive program  that  drove  the  survey.  Immediately,  a  panel  of 
the   names  of   the   professors   in  the   Operations   Research 
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department  would  appear,  and  the  judge  would  be  asked  to 
indicate  those  professors  from  whom  he  had  taken  a  course. 
After  selecting  his  own  subset  of  professors,  the  judge  was 
asked  to  rate  each  pairwise  combination  of  professors  in 
terms  of  similarity  or  dissimilarity  of  teaching  style.  If 
the  judge  had  observed  n  professors,  this  meant  that  a  judge 
would  have  to  respond  to  at  least  n(n-1)/2  prompts  for  this 
proximity  data.  For  this  study,  the  sixteen  professors  are 
identified  by  using  the  letters  A  thru  P.  The  scale 
presented  to  the  judge  ranged  from  a  value  of  1  meaning  very 
similar  to  a  value  of  9  meaning  very  dissimilar.  The  judges 
were  not  limited  to  integer  responses,  but  real  responses 
were  restricted  to  one  decimal  place.  It  was  assumed  that 
the  proximity  scale  was  an  interval  scale  meaning  the 
distance  in  similarity  or  dissimilarity  values  between  say 
values  2  and  3  was  equal  to  the  distance  in  dissimilarity 
between  values  8  and  9. 

Once  all  the  proximity  data  had  been  collected,  the 
judges  were  then  asked  to  score  the  professors  with  respect 
to  several  bipolar  scales.  In  the  case  where  students  had 
taken  a  professor  for  more  than  one  course,  the  student  was 
instructed  to  respond  to  the  bipolar  scales  based  on  the 
last  course  taught  by  the  professor.  The  scales  used  in  the 
survey  appear  in  Table  2  and  were  suggested  by  previous 
Operations  Research  students.  The  bipolar  scales  also 
ranged  in  value  from  1  to  9.  Once  each  judge  completed  the 
survey,  his  or  her  results  were  automatically  sent  to  the 
researchers  computer  disk  for  subsequent  evaluation. 

In  addition  to  the  data  collected  from  the  computer 
interactive  survey,  information  gathered  from  student 
opinion  forms  (SOF's)  were  used  to  try  to  interpret  charac- 
teristics of  the  different  professors.  Only  those  student 
opinion  forms  from  those  classes  taught  by  the  professors 
selected  in   the  interactive   survey  were   considered.    The 
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TABLE  2 
Bipolar  Scales  Osed  in  the  Compater  Sarvey 

1 .  CLASS  SIZE 

2.  THEORETICAL  VS.  APPL  I  EC 

3.  GRADING  PCL.ICY 
*.  PACE  OF  COURSE 

5.  EFFORT  REQUIRED  OUTSIDE  OF  CLASS 
5.  COURSE  RELIED  UPON  PREREQUISITES 


bipolar  scales  used  in  current  SOF  forms  appear  in  Table  3  . 
One  problem  encountered  in  the  evaluation  of  these  forms 
dealt  with  maintaining  the  purity  of  the  control  group.  For 
the  most  part,  the  SOF's  were  completed  by  the  twenty-three 
students  in  the  control  group.  However,  there  were  some 
instances  where  other  students  either  from  other  Operations 
Research  sections  or  from  other  curriculums  were  included  in 
the  evaluation  process.  The  information  collected  from  the 
SOF's  represented  different  scales  upon  which  to  measure 
teacher  performance.  The  SOF  data  was  originally  converted 
to  an  interval  scale  on  a  range  different  from  the  scales 
used         in        the        interactive        survey.  Because        linear 
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TABLE    3 
Bipolar    Scales   Used,   in   the   SOF    Forms 

I.    cauPse   ORGANIZATION 

2.  TIME   IN  CLASS  SPENT  EFFECTIVELY 

3.  INSTRUCTOP  KNO*S  WHEN  STUDENTS  OONT'T  UNDERSTAND 
MATER [ AL 

*.  DIFFICULT  CONCEPTS  MADE  UNOE RS T AND ABLE 

5.  CONFIDENCE   IN  INSTKUCI'CRS  KNC'*LECGE  IN  SUBJECT 

6.  FELT  FREE  TO  ASK  QUESTIONS 

7.  INSTRUCTOR  PREPARED  FOR  CLASS 

3.   INSTRUCTORS  OBJECTIVES  MADE  CLEAR 

9.      INSTRUCTOR  -^AOE  COURSE  WORTHWHILE  LEARNING  EXPERIENCE 

10.  INSTRUCTOR  STIMULATED  INTEREST   IN  SUEJECT  AREA 

11.  INSTRUCTOR  CARED  AEOUT  STUDENT  PROGRESS  AND  DID  HIS 
SHARE  IN  HELPING  TC  LEARN 


transformations  of  the  form  a+Lx  are  allowed  on  interval 
scales,  the  scale  used  for  the  SOF's  was  changed  to  match 
the  scale  used  in  the  interactive  survey.  In  the  case  of 
each      set    of      scales,      a      separate      regression   analysis      and 
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factor  analysis  was  done.  One  key  parameter  we  were 
interested  in  measuring  was  the  correlation  between  how  the 
judges  evaluated  the  professors  overall  performance  at 
different  times  in  the  curriculum,  SOF  data  was  only  avail- 
able through  the  end  of  the  summer  quarter  of  193U  (the  end 
of  the  judge's  sixth  quarter) .  Since  the  judges  completed 
the  computer  survey  at  the  beginning  of  the  eigth  quarter, 
only  those  professors  taken  through  the  end  of  the  sixtn 
quarter  were  evaluated.  This  meant  that  a  few  professors 
were  not  included  in  the  final  evaluation. 

C.   OUTPUT  FEOH  HDS 

The  proximity  or  dissimilarity  data  obtained  from  the 
interactive  survey  was  used  as  a  direct  input  to  a  multidi- 
mensional scaling  algorithm  KYST,  partially  developed  by 
Kruskal  £Eef-  1:  p.  1]-  The  proximity  input  values  for  this 
study  appear  in  Appendix  A.  The  output  from  the  multidimen- 
sional scaling  program  includes  visual  plots  or  spatial  maps 
depicting  the  professors  positions  in  multidimensional  space 
projected  down  to  two  dimensions  for  visual  display.  Also 
included  as  output  are  coordinates  for  each  professor  in 
multidimensioaal  space.  The  task  that  remained  was  to 
define  the  variables  that  best  explained  the  location  of 
each  professor. 

One  method  available  for  determining  what  characteris- 
tics explain  the  orientation  of  the  professors  from  the 
multidimensional  scaling  output  is  multiple  linear  regres- 
sion. The  median  value  from  each  bipolar  scale  is  regressed 
over  the  coordinate  positions  of  each  professor.  From  the 
stress  versus  dimension  curve  it  was  decided  to  concentrate 
on  a  four  dimensional  interpretation,  although  three  and 
five  dimensional  interpretations  were  considered  also.  The 
values  of  stress,   the  goodness-of-f it  function,   turned  out 


29 


to  be  0.249  for  four  dimensions  and  0.293  for  three 
dimensions.  Neither  of  these  values  indicate  a  very  good 
fit  according  to  Kruskals'  own  personal  experience.  This 
large  value  of  stress  was  an  early  indication  that  this 
linear  model  might  not  be  appropriate  in  explaining  teacher 
performance. 

Figure  3.  1  shows  the  resulting  regression  weights  or 
direction  cosines  corresponding  to  each  multiple  correlation 
for  the  four  dimensional  solution.  The  direction  cosines 
are  regression  weights  normalized  so  that  their  sum  of 
squares  equals  1.0  for  every  scale  [Ref.  3:  p.  37].  For 
example,  when  regression  weights  of  0-U178,  0.8959,  -0.1508, 
and  -0.0026  are  given  to  dimensions  1,2,3  and  U  respec- 
tively, the  multiple  correlation  between  the  resulting  coor- 
dinate  positions   and    the   respective    bipolar   scale   is    0.531. 

A  bipolar  scale  will  provide  a  good  interpretation  of  a 
dimension  when  its  multiple  correlation  coefficient  is  high. 
A   value   above    .90    is    desired.  The    values   achieved    in   this 

study  were    low.  Values   of   S^      close    to      0.5    were   typical. 

Also  a  requirement  for  good  dimensional  interpretation  is  a 
high  regression  weight  on  the  dimension  it  most  nearly 
explains.  The  results  obtained  from  the  bipolar  scales  used 
in    this   study      are   examined    in    the    next   chapter. 

D.       OUTPUT    FROM    FACTOR   ANALYSIS 

Having  looked  at  regression  modelling  as  an  approach  to 
interpreting  dimensions,  factor  analysis  was  also  considered 
a  possible  means  of  identifying  linear  factors  that  would 
help  describe  teacher  performance  from  the  data  sample.  The 
statistical  analysis  package,  SAS,  was  used  to  generate  two 
separate   factor    analysis   outputs.  One    factor    analysis   was 

completed  for  the  bipolar  scales  associated  with  the 
computer  survey  and  another  was  done  on  the  additional 
bipolar    scales   from   the    SOF    forms. 
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POSITIVE     POLES     OP               N0HMALIZE3     SEGBESSION 

COEFF ICIENTS 

BATING     SCALES 

(DIRECTION    CQSINESi 

MUL  T  IPL  = 

COMPUTER     BIPOLAR     SCALES 

01  M  1 

01  "2 

DI  "3 

0  IM4 

CORRCL AT  13N 
COEFF ICIENT 

I  . 

CLASS     SIZE 

0,3^5* 

0 .0 190 

0.91 72 

-0. 

0434 

.262 

2. 

THEOOETICAL     VS. 
APPLIED 

0.2603 

-0 .8367 

-  0.4816 

-0. 

0096 

.5  10 

3, 

GHAOING     POLICY 

0.31 97 

-0 .6784 

-0.3689 

-0. 

54  »9 

.41a 

4. 

Pace    of   COURSE 

0.2527 

-0.5951 

-0.4877 

-0. 

5865 

.453 

5. 

EFFORT     HEQUIREO 
OUTSIDE     OF     CLASS 

0.  I  3<3I 

-0.9  142 

-0.  3  25  3 

-0. 

1  975 

.3  76 

6. 

COURSE     RELIEO-UPON 
PREREQUISI TES 

0.0696 

-0  .3655 

-0.5769 

-0  . 

72  70 

.501 

SCF 

"BIPOLAR     SCALES 

7  , 

COURSE     ORGANIZATION 

0.5942 

0 .6474 

-0.4223 

0. 

22  19 

.502 

8. 

TIME     IN     CLASS     SPENT 
EFFECTI VELY 

0.64  35 

0.6  1 1  a 

-0.0025 

0. 

4597 

.493 

<3. 

INSTRUCTOR     KNOWS     «M£N 
STUDENTS     OONT     UNDER- 
STAND    MATERIAL 

0.9337 

0. 330  I 

0.01  I  3 

0, 

1375 

.235 

10. 

DIFFICULT     CONCEPTS 
MADE     UNOERST ANDA8LE 

O.a*  12 

0 .1 880 

-0.461 2 

0. 

2103 

.  4  44 

1  1  . 

CONFIDENCE      IN      INSTRUC- 
TORS    KNOWLEDGE      IN     SUB- 
JECT 

0.2928 

0. 1 632 

0.0859 

-0. 

64  04 

.095 

t  2. 

=ELT     FREE      TO     ASK 
QUEST  IONS 

0. ja72 

-0.0887 

0.04t>6 

-0  . 

1238 

.405 

13. 

INSTRUCTOR     PREPARED 
FOR     CLASS 

0. 7263 

0.5747 

-C.3012 

0. 

2265 

.'447 

1  4. 

INSTRUCTORS     OBJECTIVES 
•^AOE     CLEA« 

0.41 78 

0  .8959 

-0. 1508 

-0. 

0026 

.581 

15. 

INSTRUCTOR     MADE     COURSE 
WORTHWHILE     LEARNING 
HXPER lENCE 

0.7  0  65 

0  .42ia 

-0.4  327 

-0. 

3684 

.4   78 

16. 

INSTRUCTOR     STIMULATED 
INTEREST      IN     SUBJECT 
AREA 

0.6352 

0 .2 174 

-0.6073 

-0. 

4245 

.298 

17. 

INSTRUCTOR     CARED     A60UT 
STUDENT     PROGRESS     AND 
DID     HIS     SHARE      IN     HELP- 
ING     TO     LEARN 

0.d310 

0 .2991 

-0.4  14  3 

-0. 

21  93 

.532 

Figure  3. 1    Regression  Weights  for  the 
Four  Diaensional  Solution- 
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Initially,  a  correlation  matrix  was  produced  from  the 
raw  input  data.  Additionally,  the  common  factors  and  the 
factor  loadings  for  each  observed  variable  or  bipolar  scale 
was  produced.  Orthogonal  rotations  were  effected  to  produce 
simple  structure.  A  total  of  two  common  factors  were 
created  from  the  bipolar  scales  of  the  interactive  survey, 
and  one  common  factor  was  produced  from  the  SOF  data.  The 
specifics  of  the  factor  analysis  output  are  discussed  in  the 
next  chapter. 

E.   OUTPUT  FROM  CLUSTER  ANALYSIS 

As  a  final  measure,  a  disjoint  cluster  analysis  was 
performed  on  the  MES  data.  The  clustering  routine, 
Pastclus,  was  available  from  the  statistical  analysis 
package,  SAS.  The  number  of  clusters  into  which  the  group 
of  instructors  were  subdivided  was  specified  by  the 
researcher  to  range  from  2  to  6.  Membership  in  a  particular 
cluster  was  determined  based  upon  the  distance  from  each 
professors  position  to  the  mean  value  of  the  cluster.  The 
output  from  the  cluster  analysis  included  identification  of 
the  cluster  to  which  each  professor  belonged. 
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IV.    DATA    ANALYSIS 

A,       IHTEST    OF    THE    ANALYSIS 

The  scope  of  this  chapter  will  be  to  analyze  the  results 
obtained  from  the  completed  interactive  survey  in  the 
context  of  multiple  linear  regression  and  stepwise  regres- 
sion, factor  analysis  and  cluster  analysis.  Additionally, 
information  gathered  from  student  opinion  forms  (SOFs) ,  will 
be  evaluated  in  so  far  as  what  characteristics  or  bipolar 
scales  appear  to  have  been  most  important  in  describing 
students   perceptions    of   teacher   performance. 

An  issue  that  requires  explanation  before  the  analysis 
begins  concerns  a  vital  assumption  made  dealing  with  the 
scale  of  the  data.  Specifically,  can  an  arbitrarily  chosen 
numerical  scale  with  fixed  upper  and  lower  bounds  enable 
correct  statistical  inference  from  the  data  sample?  In  our 
study,  we  provided  tlie  judges  with  a  numerical  scale  ranging 
in  value  from  1  to  9 .  The  judges  were  allowed  to  rate  each 
professor  in  every  category  with  respect  to  this  scale.  In 
essence  we  are  imposing  an  interval  scale,  with  equally 
spaced  intervals.  Let's  consider  a  judge's  response  to  the 
question   of   grading    policy.  It    may      be    that    the    judges    can 

rate  the  different  objects,  in  our  case  professors,  at  best 
on  an  ordinal  scale.  The  judge  may  be  able  to  say  that 
professor  A  is  a  harder  grader  than  is  professor  B,  but  not 
how  much  harder.  ^e  are  assuming  that  the  judges,  when 
responding  to  these  bipolar  scales,  realize  that  we  are 
infering   an   interval      scale    on    their    responses.  Fe    assume 

that  when  they  complete  the  survey,  that  they  realize  that 
each  integer  value  on  the  scale  divides  the  scale  into  e^ual 
intervals.         We    assume   that    a    judge    will    rate  each    professor 
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knowing  that  the  distance  between  say  a  score  of  3  and  4  is 
equal  to  the  distance  between  a  score  of  5  and  6.  We  find 
ourselves  doing  this  in  order  to  follow  the  example  of 
Kruskal  as  closely  as  possible  and  because  the  statistical 
methods  used  to  evaluate  the  data  require  at  least  an 
interval  scale.  Given  that  we  are  assuming  an  interval 
scale,  we  feel  comfortable  in  transforming  the  data  with  any 
linear  transformation  of  the  form  a  +  bx.  The  major  point 
being  made  here  is  that  the  researcher  should  keep  in  mind 
the  scale  of  the  data  when  interpreting  the  significance  of 
the  output. 

B.   "HHAT  DID  WE  LEARB  FBOM  MDS? 

As  mentioned  in  Chapter  II,  multidimensional  scaling 
attempts  to  determine  the  structural  relationships  between  n 
objects  from  a  matrix  or  halfmatrix  of  proximity  data.  A 
major  result  of  the  process  is  the  spatial  mapping  of  the  n 
objects,  usually  projected  down  to  the  planes  of  each  pair 
of  dimensions  for  easy  visual  interpretation.  The  final 
configuration  of  points  represents  the  best  fit  of  the  n 
objects  according  to  the  stress  criterion.  Table  4  contains 
the  final  configurations  for  all  sixteen  professors  in  four 
dimensions. 

In  order  to  determine  which  dimensionality  scheme  best 
suggests  the  characteristics  that  set  the  professors  apart 
from  one  another,  it  helps  to  review  the  spatial  maps 
created  from  the  data.  Appendices  3  and  C  contain  the 
spatial  plots  of  the  sixteen  professors  for  four  and  five 
dimensional  solutions.  Each  pairwise  combination  of  axes  is 
plotted  against  each  other-  This  spatial  orientation  can 
sometimes  suggest  from  mere  inspection  those  characteristics 
that  cause  some  professors  to  be  more  alike  than  others. 
However,    we  do   not  usually   rely   upon  visual   inspection 
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TABLE    4 

Final   Configaration    for    the    16    Professors 

in    4    Dimensions 


PROF 

^■S  SO'"^ 

Dlf-.     I 

DIM.     2 

C!M.    3 

DIM.     ^ 

L. 

A 

-0.737 

0.2  49 

0.432 

0 .  15  0 

2. 

B 

1.2C9 

0.  069 

-0.517 

0.172 

3. 

C 

-0.359 

-0.207 

-0.430 

0.094 

4. 

D 

-0.648 

-0.382 

-0.093 

0  .  666 

5, 

r: 

-0.724 

-0.5o9 

0.314 

-0.113 

6. 

F 

0.639 

0,267 

0.042 

-0.056 

7. 

G 

-0.706 

-0.085 

-0. 154 

-0.416 

8. 

H 

0.450 

-0.574 

0.265 

-0.279 

9. 

T 

-0.310 

0  .  6  42 

-0.006 

0.434 

10. 

J 

-0.252 

0.3  64 

0.312 

-0.336 

11. 

K 

1.328 

-0.205 

0.197 

-0.030 

12. 

L 

-C.923 

-0.2  23 

-0.311 

-0.370 

13. 

M 

0.773 

0.391 

0.775 

-0.113 

14. 

M 

-0.047 

-0.254 

0.477 

0.381 

15. 

D 

-C.159 

1-203 

-0.611 

-0.104 

16. 

P 

0.517 

-0.693 

-0.743 

-0.G30 
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alone.  Very  often  a  plot  indicating  how  stress  improves  as 
a  function  of  dimension  gives  the  dimensional  interpreta- 
tion. Figure  4.1  is  such  a  plot  for  the  data  collected  in 
this  study.  It  can  be  seen  that  a  one  dimensional  interpre- 
tation yields  a  very  high  stress  value  of  0.463.  This  would 
suggest  that  a  one  dimensional  interpretation  alone  would  be 
inappropriate.  We  would  like  a  noticeable  elbow  to  occur  in 
the  stress  vs.  dimension  plot,  for  this  normally  indicates 
the  most  suitable  level  of  interpretation.  We  observe  that 
this  plot  does  not  exhibit  the  noticeable  elbow.  Instead, 
the  slope  of  the  curve  decreases  gradually  and  we  are  left 
to  look  for  other  means  to  help  determine  dimensionality. 

Kruskal  and  Wish  offer  an  alternate  method  or  rule  of 
thumb  in  choosing  dimensionality  [ Ref -  3  p.  34].  They 
suggest  that  the  number  of  stimulus  objects  minus  one,  in 
our  case  fifteen,  should  exceed  four  times  the  dimension 
chosen  for  interpretation.  This  would  offer  a  choice  of 
dimension  no  greater  than  3.75.  They  caveat  this  statement 
by  saying  that  this  rule  has  only  been  found  to  hold  for 
three  dimensions,  and  that  further  study  is  needed  to  see  if 
it  is  appropriate  for  higher  dimensions  as  well.  We  chose 
to  emphasize  a  four  dimensional  interpretation,  however  a 
three  and  five  dimensional  interpretation  were  considered 
also. 

C.   TBE  USE  OF  REGRESSION 

Once  the  choice  of  dimensionality  had  been  made,  the 
next  task  was  to  determine  which  characteristics  represented 
those  dimensions  best.  To  do  this,  we  decided,  as  do 
Kruskal  and  others  [fief.  3;  pp.  35-36]^  to  use  multiple 
linear  regression  as  a  means  of  clarifying  this  issue. 
Again,  we  reiterate  that  a  separate  regression  analysis  was 
conducted  for  the  scales  used  in  the  computer  survey  as  well 
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Figure   4,  1        Stress    ts-    Dimension   Plot 
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as  the  scales  obtained  from  the  SOF  forms.  Specifically,  we 
regressed  the  median  value  of  each  bipolar  scale  over  the 
coordinates  of  each  professor,  and  recorded  a  multiple 
correlation  coefficient.  If  the  value  of  the  multiple 
correlation  coefficient  was  high,  i.e.,  close  to  1,  then  we 
felt  inclined  to  believe  that  this  scale  was  important  in 
distinguishing  professors.  If  the  value  of  the  multiple 
correlation  coefficient  was  low,  i.e.,  close  to  0,  then  that 
particular  scale  was  not  perceived  as  being  important. 

Tables  5,  6  and  7  illustrate  the  two  sets  of  bipolar 
scales  used  in  the  study,  the  corresponding  normalized 
regression  weights  or  direction  cosines,  and  the  multiple 
correlation  coefficients  associated  with  each  scale  for 
dimensions  3,  U  and  5.  It  can  be  seen  that  for  the  most 
part,  the  multiple  correlation  coefficients  hover  near  0.5 
and  typically  increase  as  the  the  dimension  increases. 
Figure  4.2  depicts  the  changes  in  the  multiple  correlation 
coefficient  for  each  bipolar  scale  as  the  dimension  for 
interpretation  changes-  In  the  table,  bipolar  scales  1  thru 
6  refer  to  the  scales  from  the  computer  survey  and  bipolar 
scales  7  thru  17  refer  to  those  scales  from  the  SOF  form. 

Since  none  of  the  scales  exhibited  truly  high  correla- 
tion coefficients,  we  chose  those  that  had  the  highest 
values  and  used  them  to  interpret  the  dimensions  in  our 
study.  The  regression  analysis  on  the  computer  survey 
scales  suggested  that  these  scales  were  inappropriate  indi- 
cators for  this  set  of  judges.  For  the  four  dimensional 
analysis,  we  see  that  the  bipolar  scales  'theoretical  vs. 
applied  course'  and  'course  relied  upon  prerequisites'  have 
the  highest  correlation  coefficients  for  that  group  of 
scales.  Specifically,  the  correlation  coefficients  are  .510 
and  .501  respectively.  The  scale  'applied  vs.  theoretical 
course'  loads  heavily  on  dimension  2  with  a  normalized 
regression  weight  of  -0.8367.    Thus,  it  would  seem  that  the 
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TABLE    5 

Ltiple   Corre 

DiotensioDS 


Regression  Weights  and  Multiple  Correlation 
Coefficients  for  tne  Bipolar  Scaxes  in  3  Diaeas: 


CC-1PUTEP  eiPGuAP  sc;,Lf:s  oimi  oi.z  Ol-i     EcEpiici^H^ 

1.       CLASS    SIZE  -0.3830  0.012^  0.9237  .2^5 

'•      irPmV-^^''    '''  -'-''''  -C-^-^  -^-^i^-  .557 

i.       GRACING    POLICY  -0.3794  -C.oOOS  -C.7039  .313 

^.       PACc    QF     COUR-2  -0.2970  -C.5L41  -0.80W  .338 

'•       oCTil5:-'a,i^<?[-f?c  -"3-^553  -0.7963  -0.5346  .313 

'•       PRLRJL?lKir''''  -'''''''  -^-^054  -0.9483  .329 

scF  eiPCLJfv  scal:es 

7.       COURSE    OPGANIZATIDN  -0.6108  C. 7432  -0.2731  ^76 

'•       EFFyv'vi[:^^-     ^P^''^  -^-^^^^  C.6238  0.2464  .'428 

'•       iTaal'^n°^CN'^"S;^C.^.^-'    -'-''''  '-'^'^  a.i098  .235 
SlAfiD  MaTcRlAL 

IC.     DIFFICULT    CaNCEPTS  -0.6633  a    ;'9^I  -n    a  1  ,  =: 

^'AOE    UNDEiTSTANOAeLE  ^.<^«'->J  0.2921  -0.4115  .-42 

li.     CnNFTDE'iCS     IN  -0.2873  n    nr.os  n    oc^-r 

IMSTRur.TG^S    KMQWL^TIGE  '^•""^'^  0.0098  0.9577  .076 
lU    SUBJIEC" 

^^'     Hkhwii    ^°    ^^'  -0.9975  -C.0345  0.0614  .394 

''*     FCrcus?'    '"^^-■■"^C  -^-^5.7  G.6372  -C.l4ol  .407 

14.  IfJSTPUC^CaS   08JEC-  -0.42t4  C   aqfl^  -    1  1  •  ->  c  ^o 
TIVE3    :i\0£    CLEA;^  u.^^^t^  l,.898J  V-.1W2  .503 

15.  INSTRUCTOR     MADE  -0.7098  C    "^^S^  -,t    -<aia  ,      -, 
CGURSE    WGSTHWHILE  ^-'U-^^a  C.5354  -0.3913  .463 

LEARNING    cXPERIZNCE 

'''     ffl?  =  RgT°?N'I5^!j^^T'°  -'-^"'^  ^•^'^^^  -<^-539  .289 
AREA 

17.     INSTRUCTOR    CARHC  -0.3042  0    464'?  -o    -kym  = -,-, 

ASnuT    STUCrZN^    =PCG?=53    '"'''^^'^  U.-645  -C-3707  .527 
ANO    CIO     HIS    SHARE    IN 
HGLPI'JG    TO     LEA",N 
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fact  that  a  course  was  perceived  as  being  theoretical  or 
applied  most  closely  describes  the  effect  of  the  professors 
ultimate  positioning  with  respect  to  dimension  2.  Likewise, 
the  scale  ^course  relied  upon  prerequisites'  loads  heavily 
on  dimension  4  with  a  regression  weight  of  -0.7270.  This 
appears  to  suggest  that  dimension  4  is  most  nearly  explained 
by  how  the  judges  perceived  how  each  course  relied  upon 
other  prerequisite  courses.  The  regression  weights  have  a 
geometrical  interpretation  as  the  cosine  of  the  angle 
between  the  dimension  upon  which  it  loads,  and  the  associ- 
ated scale,  Unfortunately,  the  two  scales  with  the  next 
highest  multiple  correlation  coefficients,  'pace  of  course' 
and  'grading  policy',  load  heavily  upon  dimensions  2  and  4 
as  well.  They  probably  contribute  to  the  explanation  of 
those  dimensions  also.  Not  one  of  the  scales  with  a  high 
multiple  correlation  coefficient  loads  heavily  upon  dimen- 
sions 1  or  3,  These  dimensions  are  left  unexplained  and  new 
scales  are  needed  to  determine  an  appropriate  explanation. 

A  similar  analysis  was  conducted  for  the  bipolar  scales 
used  in  the  SOF  forms.  Here  the  scales  with  the  highest 
multiple  correlation  coefficients  were  1)  instructors  objec- 
tives made  clear;  2)  instructor  cared  about  student  progress 
and  did  his  share  in  helping  to  learn;  3)  course  organiza- 
tion and  4)  time  in  class  spent  effectively.  Again,  the 
same  problem  occurred  here  as  in  the  analysis  of  the 
previous  set  of  scales.  Almost  all  those  scales  with  the 
highest  multiple  correlation  coefficients  load  heavily  on 
only  two  of  the  four  dimensions,  namely  dimension  1  and  2- 
Dimension  1  appears  to  convey  information  about  how  each 
professor  cares  or  interacts  with  his  students  on  a  personal 
level.  The  bipolar  scales  with  high  multiple  correlation 
coefficients  that  load  heavily  on  this  dimension  are,  1) 
instructor  knows  when  students  don't  understand  the 
material;  2)    difficult  concepts  were  made  understandable;  3) 
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TABLE  6 

Heqression  Weights  and  Multiple  Correlation. 
Lficients  for  tie    Bipolar  Scales  in  4  Dimension: 


Coer 


POSITIVE    POLcS    OF  MCP.MALIZFn    R=r,seS?!CN    CCFFF1CIFMT3 

RATING    SCALES  (DIRtCTIQN    COSlUtS)  ^ULTTPL' 

cn=.-'=L  i'rT cr: 

CCMPUTER    eiPOLAR    SC-iLES  OJML  DI,"<2  DIM3  OIM^      CD2r  F IC I  E.N'' 

1.  CLASS    SIZE  0.3954         O.OL90         0.^172      -0  .^'♦34  .262 

2.  THEOS^TICJL    VS.  0.2603       -C.3Jo7       -a.4dL6       -0.J096  .51C 
APPLIED 

3.  GRADING     PCLICY  0.3197       -0.6734       -0.3689       -0.5489  . 'V  L  ^ 

4.  PACE    OF    COURSE  0.2527      -G.5951       -0.4377      -0.5J165  .453 

5.  EFFCRT    R=(1UIRED  0.I39L       -0.9L42      -0.3253       -0.1.975  .376 
QUTSIDF.    a-    CLASS 

6.  CnuRSF    RELIPD-'.JPCN  0.0696       -0.3655       -0.57o9       -0.7270  .50  1 
PRtSEQUI  SfES 

SOF     BI  POLAR     S:aU:S 

7.  COURSE    ORGANIZATION  0.5942  0.6474      -0.4223  0.2219  .502 

8.  TIME     in     CLASS     SPENT  0.6435  CollS       -0.J025  0.4597  .493 
EFFECTIVELY 

9.  INSTRUCTOR     KWOWS    WHEN       0.9337  C.3301  0.0113  0.1375  .285 
STU05MTS    3GNT    UNCES- 

STANC    '-lATtPrAL 

IC.     DIFFICULT    CONCEPTS  0.3412         0.1330      -0.4612         0.2103  .4.^4 

fAOE    UTiOfcdSTiNCAeLS 

11.  CnNFID'=MCl!     !M     INSTP.UC-    0.2928  C.1332  0.6859       -O.o404  .095 

TCPS    KNOWLECGc    IN    JUe- 
JFCT 

12.  FEL^    FREE    TO    ASK  0.9872       -0.0837  0.0466       -0.1238  .405 
QUHSTIONS 

13.  PL^'.TRUn'aa    prepared  0.7263  0.5747       -0.3012  0.2265  .447 
FCR    CUSS 

14.  IMSTRUCTC:\S    OBJECTIVES    0.4178  C.3959       -0.1508      -0.0026  .581 

•'ADE    CLE  AS 

15.  inSTRUCTQ^    HAOe     COURSE    0.7065  C.4213       -0.4327       -0.3684  .473 
ktORTHWHILE     LEARNING 

EXPERIc.NCe 

16.  IMSTlUC~Ot^     STIMULATED       0.6352  C.2174      -0.6073      -0.4245  .298 
iriTEREST     IN    SUBJfcCT 

4R£A 

17.  IfJSTRUCTOn    C-R-IO    A3GUT    0.3310  0.2991       -0.4143       -0.2193  .532 
STUCEN"^     PICGRESS     iNC 

010    HIS     SHAPC     IN    HC-LP- 
iriG    TO    LE^RN 
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felt  free  to  ask  questions;  U) instructor  made  course  a 
vorthwhile  learning  experience  and  5)  instructor  cared  about 
student  progress  and  did  his  share  in  helping  to  learn. 
Dimension  2  seems  to  describe  information  concerning  how 
instructors  are  perceived  as  being  organized  or  prepared  for 
their  course.  The  scales  vith  high  multiple  correlation 
coefficients  that  load  heavily  upon  this  dimension  are  1) 
course  organization;  2) time  in  class  spent  effectively  and 
3)  instructors  objectives  are  made  clear. 

In  addition  to  the  bipolar  scales  used  for  the  interac- 
tive survey  and  the  SOF  forms,  each  judge  was  asked  to 
provide  a  rating  of  overall  teaching  performance  for  each 
professor.  The  median  values  for  each  professor  were 
obtained  and  are  tabulated  in  Table  8  .  Although  we  have 
kept  the  analysis  of  the  interactive  survey  separate  from 
the  SOF  data,  we  were  interested  in  the  correlation  between 
professor's  overall  performance  judged  on  two  separate 
occaisions.  The  first  occaision  occurred  when  the  judge 
completed  the  SOF  form  for  the  course  taught  by  the 
professor.  The  second  occaision  was  in  conjunction  with  the 
interactive  survey.  The  correlation  between  ratings  of 
overall  performance  for  the  professors  was  .58,  lower  than 
expected.  Suggestions  as  to  why  this  correlation  is  low 
include  the  effect  of  time-  Judges  may  be  less  able  to 
evaluate  a  particular  instructors  overall  performance  as 
time  goes  by.  Also,  students  may  change  their  opinions  of 
teachers  overall  performance  after  having  seen  a  professor 
teach  a  variety  of  different  classes.  For  example,  a 
student  who  enjoys  applied  courses  might  rate  a  professor 
differently  after  having  had  him  f or  a  Stochastics  models 
course  than  he  would  for  say  an  applied  course  like  Test  and 
Evaluation.  In  any  event,  we  did  expect  the  correlation  on 
overall  performance  to  be  higher  than  it  turned  out  to  be. 
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TABLE  7 

Regression    Weights    and    Multiple    Correlation 
Coefficients    for   tlie    Bipolar    Scales    in    5    Dimensions 
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stepwise  regression  was  utilzed.  in  order  to  determine 
which  scales  or  independent  variables  explained  the  greatest 
proportion  of  variance  in  teacher  performance.  The  API 
function  STEEG,  was  used  to  compute  the  order  in  which  the 
independent  variables  were  to  enter  into  the  regression 
model.  The  output  from  STREG  is  displayed  in  Table  9  for 
both  the  six  scales  asked  in  the  computer  survey  as  well  as 
the  scales  from  the  SOF  forms.  The  output  indicates  the 
order  in  which  the  variables  entered  the  model,  the  coeffi- 
cients associated  with  each  independent  variable,  the  stan- 
dard error  associated  with  each  coefficient,  the 
corresponding  t- value  and  the  proportion  of  the  variation  of 
the  dependent  variable  accounted  for  by  the  independent 
variable.  We  see  that  the  coefficient  of  multiple  determi- 
nation,  R2,  is  low  for  the  six  bipolar  scales  used  in  the 
computer  survey.  In  all,  these  scales  account  for  only  13.8 
percent  of  the  variation  in  the  dependent  variable,  overall 
instructor  performance.  However,  the  stepwise  regression 
performed  on  the  eleven  bipolar  scales  from  the  SOF  forms 
indicate  a  coefficient  of  multiple  determination  of  .986. 
This  set  of  scales  explains  more  of  the  variation  in  the 
judges  responses  to  overall  instructor  performance  than  does 
the  previous  set  of  scales.  In  particular,  'course  organi- 
zation' appears  to  have  a  significant  impact  on  students 
perception  of  teacher  performance  since  it  accounts  for  over 
73  percent  of  the  variation  alone. 

One  problem  that  concerned  us  dealt  with  the  fact  that 
for  the  SOF  data,  even  with  a  value  of  R2  equal  to  .986, 
most  of  the  independent  variables  were  statistically  insig- 
nificant, Even  the  scale  'course  organization',  which 
accounted  for  73  percent  of  the  variation  in  overall 
performance  ,  proved  to  be  statistically  insignificant  at 
the  0.05  level.  A  high  value  of  R2  and  statistically 
insignificant   variables   usually   is   symptomatic   of   high 
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Figure  4.2   Changes  in  the  Value  of  the  Multiple  Correlation 
Coefficient  as  a  Function  of  Dimension. 
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mult icollinear ity,  Multicollinearity  simply  means  that  one 
or  more  so  called  independent  variables  are  highly  corre- 
lated with  another  independent  variable  or  is  a  linear 
combination  of  a  number  of  the  independent  variables.  The 
problem  with  high  multicollinearity  is  that  the  estimates 
for  the  regression  coefficients  become  unreliable  from  one 
sample  to  the  next.  Cur  confidence  in  our  ability  to  deter- 
mine the  effect  of  an  independent  variable  withers.  To  show 
that  high  multicollinearity  exists,  we  need  to  regress  each 
independent  variable  over  all  the  other  independent  vari- 
ables to  see  if  any  are  a  linear  combination  of  the  others. 
Table  10  shows  the  values  of  E^  obtained  by  regressing  each 
independent  variable  over  the  others  for  both  models. 

Several  options  are  available  when  confronted  with  high 
multicollinearity  as  we  were  in  this  study.  One  solution  is 
to  increase  the  sample  size.  This  turned  out  not  to  be  a 
useful  alternative  since  our  sample  size  was  fixed.  Another 
strategy  is  to  combine  several  variables  that  are  highly 
correlated  into  a  single  indicator  as  long  as  it  makes 
sense.  This  is  possible  for  several  scales  on  the  SOF  fora 
which  are  highly  correlated.  A  third  alternative  is  to 
discard  those  variables  which  are  linear  combinations  of  the 
others  and  are  the  cause  for  the  high  multicollinearity. 
After  discarding  the  offending  variables,  a  new  regression 
equation  can  be  created  and  a  check  for  statistical  signifi- 
cance made  anew. 

D.   BESOITS  OF  FACTOB  18ALISIS 

Factor  analysis  supposes  that  some  common  factors 
smaller  in  number  than  the  originally  observed  variables, 
account  for  the  covariation  of  the  originally  observed  vari- 
ables. Factor  analysis  assumes  a  linear  causal  relationship 
similar  to  linear  regression  analysis.    For  this  study,  two 
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TABLE  9 
Output  from  Stepwise  Regression  Procedure 


ofeSLiLfs   FROM    sTeo«(se    seGBessiQN    gf    aiPOUAs    screes  from    iNrePAcrtve 

ScnvHY     QN      INSTPUCTOR     QVESALL     PE  FiF  CP  M  ANCc  . 

X     OF 

QPOe«      IN     «MtCH     VAPIABLE  seCaeSSICN  STA^OASO  T-VALUt         VAPlAftON 

eMTFPeo     THE     MOOeL  coefficient  EOKOB  EXPLAINED 

rHEOPEriCAi.     vs.     APPUIEO  0.21*  0.26?  0.5d4  0.04* 

GPAOING     POLICV  -0.517  l.AOl  -0.368  0.016 

EFFOPr  QEOUIPEO  QUTSIOE         0.485         0.6*7  -0.750       0.02* 

CLASS 

PACE     OF     CauPSE  2.392  3.225  0.742  0.012 

CCUPSE     PEHEO     UPON     PPE-  -0.483  0.844  -0.573  0.041 

PEQUISI THS 


CLASS     SIZE  0.028  0.655  0.042  0  .  00  I 

X     OF      INSTPUCTOR     GVEPALL     PEHFORMaNCE     EXPLAINFO     av     SCAuES  0.139 

RESULTS     FPQM     STEPWISE     SEGRESSICN     OF     aiPOLAS     SCALES     FROM     SQF    FORMS 
ON     INSTRUCTOR     OVERALL     FEKFQRManCE 

X     OF 
ORDER      IN     V»M  I  CM     VARIABLE  HEGPESStON  STANOARQ  T-VALUE         VARIATION 

ENTERED     THE     MODEL  COEFFICIENT  EPROW  EXPLAINED 

COURSE     ORGANIZATION  1.070  0.411  2.605  0.737 

INSTRUCTOR     KNOWS     aMEN  1.234  0.22'*  5.498  0.067 

STUDENTS     UNDERSTAND 
MA  TER  I  AL 

FELT     FREE     TO     ASK     QUES-  -1.819  0.339  -5.363  0.036 

T  IONS 

CONFIDENCE      IN     INSTRUCTORS  2.062  O.'«40  4.691  0.031 

•cnObLEOGE      in     SUBJECT 

INSTRUCTOR     STIMULATED      IN-  -0.415  0.416  -0.996  0.015 

TEPEST      IN     SUBJECT     AREA 

TtMF      IN    CLASS     SPENT     EFFEC-        -1. 000  0.^50  -2.859  0.031 

Tl VELV 

INSTRUCTOR     CARED     ABOUT     S T U-         1.539  0.629  2.444  0.020 

OEMT     PROGRESS     AND     DID     HIS 
SHARE      IN     HELPING     TO     L£AHN 

INSTRUCTOR     MADE     COURSE  -1.795  0.679  -2.041  0.0J3 

WORTHiuMtLE     LEARNING 
EXPEQ  lENCE 

INSTRUCTOR  PREPARED  FOR         0.679         0.400         1.696       0.004 
CL'»SS 

DIFFICULT     CONCEPTS     -AOE  0.575  0.3*0  1.691  O.Oll 

UNDERSTAND ABLE 

INSTRUCTORS  OBJECTIVES  MADE    0.107         0.227         0.470       O.OOl 
CLEAR 

X     OF      INSTRUCTOR     OVERALL     PERFORMANCE     EXPLAINED     BY     SCF     SCALES        0.986 
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TABLE  10 
Table  of  Values  for  R2  Due  to  High  Multicoilinearity 


4     ChPCK     PQf     "UT  ICQLL  t  NEAR  I  TY     AMONG      INOePGNOeNr     VAWJAOUSS     neSUL''tO      tN      THe 
PQLLOwInG     values     of     O-5GUAOe0. 

CQMPureR  aiPOL4W  scales 

oe°eNoeNr  vAPiABLt  besultisg  a-sQu»«eo 

CLASS  SIZE  0.526 

rHeo«»eTiCAL  vs.  applieo  0.2*5 

GRADING     POLICY  0.951 

oaCE     UF    COURSE  0.^7a 

EFPQPT     REQUlseO     QurSIOE     CLASS  0.826 

CQUWSe     RELIED     UPON     PREREQUISITES  0.391 


SOF     =ilPQLAR     SCALES 

OE°E"JOENr     VARIA8I.E  RESULTING     R-SCUAREO 

COURSE      QRGANt^ArtON  0.96! 

Tiwe      IN    CLASS     SP«;NT  EFFECTIVELY  0.9*1 

IISSTRuCTOR     KNOWS     .HEN     STUDENTS     CON  •  T  0.905 

UNDERSTAND     MATeniAL 

OIFFICULT    CONCEPTS     MADE     UNOE R S T ANO ABLE  0.9*7 

CONFIDENCE      IN     INSTRUCTORS     XNQoLEOGE      IN  0.762 

SUBJECT 

FELT     FREE     TO     ASK     QUESTIONS  0.9*1 

INSTRUCTOR     PREPARED     FOR     CLASS  0.963 

INSTRUCTORS     OBJECTIVES     "ACE     CLEAR  0.8*8 

INSTRUCTOR     v«AOe     COURSE      -OfiTHwMlLE  0.986 

LEARNING  Experience 

INSTRUCTOR  STIMULATED   INTEREST   IN  0.930 

SUBJECT  AREA 

INSTRUCTOR  CAWED  ABOUT   STUDENT  0.980 

PROGRESS  AMD  DID  HIS  SHAOE   IN 
HELPING  TO  LEARN 
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separate  factor  analyses  were  conducted,  one  ol  each  set  of 
bipolar  scales.   Each  will  be  discussed  separately. 

The  factor  analysis  conducted  on  the  six  bipolar  scales 
used  in  the  computer  survey  yielded  the  results  shown  in 
Table  11  .  In  addition  to  the  correlation  matrix,  the  unro- 
tated  factor  loadings  and  the  orthogonally  rotated  factor 
loadings  appear  in  the  table.  Variables  with  factor  load- 
ings that  are  close  numerically  suggest  a  common  interaction 
or  measure.  It  seems  as  though  two  common  factors  are 
present  in  the  six  variables  based  upon  the  factor  loadings. 
The  first  factor  explains  44  percent  of  the  variation  in  the 
data.  This  factor  seems  to  combine  the  affect  of  'grading 
policy',  'effort  required  outside  class',  and  'pace  ox 
course'.  The  second  factor  appears  to  be  a  composite  effect 
of  'class  size'  and  'course  relied  upon  prerequisites'.  The 
scale  'class  size'  has  a  high  positive  factor  loading 
whereas  the  scale  'course  relied  upon  prerequisites'  has  a 
fairly  high  negative  loading.  This  would  seem  to  make  sense 
since  those  classes  taken  early  in  the  curriculum  tended  to 
be  large  and  the  earlier  courses  did  not  usually  require  a 
significant  amount  of  prerequisite  courses. 

The  results  of  the  factor  analysis  on  the  SOF  data 
yielded  the  correlation  matrix  and  factor  loadings  in  Table 
12  .  From  this  factor  analysis,  we  see  that  only  one  common 
factor  accounts  for  the  variance  in  the  data.  This  is 
reasonable  since  most  of  the  bipolar  scales  from  the  SOF 
forms  are  highly  correlated.  This  factor  could  describe  the 
student-professor  interaction  effect  discussed  earlier. 

Figures  4.3,  4.4,  4.5  illustrate  boxplots  of  factor 
scores  for  each  professor  for  all  three  factors.  Factor  1 
again  is  an  indicator  of  the  combined  effect  of  'grading 
policy',  'pace  of  course',  and  'effort  required  outside 
class'.  It  might  be  condensed  into  a  general  workload  index 
with  a   high  factor   score  indicating   those  professors   for 
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TABLE  11 
Table  of  Factor  Analysis  Results  on  Survey  Data 
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SCiLE  3=  oPftOlNG     POLICE 

sc&(_e  4=  PACE    CF   cau»sE 
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TABLE    12 
Tatle   of   Factor   Analysis   Results   on   SOF    Data 


SQFl^CauSSE     QHGANtZArtCN 

S0P2=r[Me      IN     CLASS     SPENT     cPPftCTlveUt 

SOF  3=  1  MS  rsuCTOR     KNO*S      aHgN     STLOE'^rS     OONT     UNCERSTANO     maTE^IAU 

SQF«i  =  0  IFF  ICUL  r     C0NC£OTS     «AOe     UNCE  R  S  T  ANO  AS  LE 
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whom  the  workload  was  a  concern  to  the  judges.  Professors 
B^  K,  K,  L,  and  P  all  have  high  median  factor  scores  with 
respect  to  this  common  factor.  Factor  2  might  represent  a 
composite  effect  between  the  scales  'class  size'  and  'coarse 
relied  upon  prerequisites'.  Professors  B,  I  and  N  had  high 
factor  scores  with  respect  to  this  indicator.  Factor  3 
encompasses  a  general  student-professor  interaction  effect. 
Most  of  the  professors  had  the  same  spread  of  factor  scores 
except  for  professor  K.  Here  the  judges  exhibited  a  higher 
level  of  controversy  for  this  characteristic. 

E-   EESUITS  OF  CLOSTEB  ANALYSIS 

A  final  means  of  looking  at  the  groupings  of  professors 
focused  on  a  cluster  analysis.  In  this  technigue, 
professors  who  were  perceived  to  be  similar  to  one  another 
were  grouped  together  in  the  same  cluster.  Other  professors 
were  likewise  grouped  in  other  disjoint  clusters  so  that  no 
professor  belonged  to  more  than  one  cluster.  The  professors 
were  assigned  to  clusters  based  upon  their  interpoint 
distance  from  the  cluster  means.  An  initial  cluster  seed  is 
selected  and  the  iterative  process  continues  until  the 
observations  become  stable,  i.e.,  each  observation  settles 
down  into  a  steady  state  cluster.  It  remains  for  the 
researcher  to  decide  upon  the  number  of  clusters  that  best 
describe  the  groupings  of  professors.  One  strategy  that  was 
employed  in  determining  the  appropriate  number  of  clusters 
resulted  from  interviews  with  the  students.  Tables  13  and 
1U  display  the  clusters  to  which  each  professor  belonged 
from  the  four  and  five  dimensional  solutions. 
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Figure   4.3        Boxplot    of    Factor    Scores    for   Factor    1 
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Figure  4.4    Boxplot  of  Factor  Scores  for  Factor  2 
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Figure  4.5        Boxplot   of   Factor    Scores   for   Factor   3 
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TABLE  13 
Taile  of  Clusters  for  Four  Diaeasional  Solution 
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TABLE  14 
Tai>le  of  Clusters  for  Five  Dimensional  Solution 
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V.  SOMMABY  OF  RESULTS 

A.   BEVIEW  OF  THE  ESSENTIAL  ITEMS 

The  purpose  of  this  study  was  to  uncover  information 
about  students  perceptions  of  teacher  perforaance  from  data 
gathered  via  a  computer  interactive  survey  and  SCF  fcras. 
The  control  group  for  the  interactive  survey  was  the 
Operations  Research  section  graduating  in  ilarch  of  1DC5. 
The  data  consisted  of  proximity  information,  how  similar  or 
dissimilar  professors  were  perceived  as  being,  and  rankings 
on  bipolar  scales-  Bipolar  scales  were  chosen  from  SOF 
forms  and  suggestions  from  previous  Operations  Research 
students.  Separate  analyses  were  conducted  on  both  sets  of 
scales.  The  level  of  the  data  was  assumed  to  be  interval 
scale  in  order  to  utilize  the  statistical  methods  involved 
in  the  analysis. 

Multidimensional  scaling  was  used  as  the  primary  means 
of  evaluating  characteristic  differences  among  professors. 
A  monotone  relationship  among  similarity  data  was  the 
primary  constraint  used  in  determining  a  final  spatial 
mapping  of  professors  coordinate  positions  in  multidijien- 
sional  space.  The  goodness-of -f it  criterion  used  to  measure 
the  degree  to  which  the  data  conformed  to  the  aonotonicity 
requirement  is  known  as  stress.  In  essence,  a  value  of 
stress  between  .05  and  .1  would  indicate  a  very  good  fit. 
Unfortunately,  the  stress  value  associated  with  a  four 
dimensional  interpretation  of  our  data  was  .242,  indicating 
a  less  than  good  fit.  Other  methods  helped  juide  the  choice 
of  dimensionality. 

Aside  from  visual  inspection  of  the  spatial  mappings, 
multiple  linear   regression  analysis   was  used   to  determine 
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the  most  important  characteristics  that  appeared  to  set  the 
professors  apart  from  one  another.  This  was  done  by 
regressing  the  median  values  of  the  bipolar  scales  over  the 
coordinate  positions  of  each  professor  in  four  space, 
obtained  by  the  MDS  program,  KYST.  The  choice  of  the  most 
important  characteristics  follows  from  the  bipolar  scales 
with  high  multiple  correlation  coefficients.  The  multiple 
correlation  coefficients  associated  with  all  bipolar  scales 
vere  lower  than  what  was  hoped  for.  The  correlation  coeffi- 
cients were  typically  around  .5.  For  those  characteristics 
deemed  important,  high  regression  weights  determined  exactly 
which  dimension  the  associated  scale  most  nearly  repre- 
sented. For  the  bipolar  scales  used  in  the  interactive 
survey,  the  scales  'applied  vs  theoretical  course*  and 
'course  relied  upon  prerequisites'  proved  to  have  the 
highest  correlation  coefficients.  These  scales  most  nearly 
explained  dimensions  2  and  U.  The  other  two  dimensions  were 
more  difficult  to  explain  since  no  scale  with  a  high 
multiple  correlation  coefficient  loaded  heavily  on  them. 

For  the  set  of  scales  obtained  from  the  SOF  forms,  a 
similar  result  occurred  with  the  scales  having  high  multiple 
correlation  coefficients  loading  heavily  on  only  two  dimen- 
sions. The  scales  with  the  highest  correlation  coefficients 
here  were  1)  instructors  objectives  made  clear,  and  2) 
instructor  cared  about  student  progress  and  did  his  share  in 
helping  to  learn.  There  seemed  to  be  two  indicators  coming 
from  the  SOF  forms.  One  indicator  seemed  to  focus  on 
instructor  organization  and  preparation.  The  other  indi- 
cator involved  a  student-instructor  interaction  effect. 
Basically,  how  did  the  judges  perceive  the  instructor  as 
caring  about  the  students  progress  in  the  course?  This 
effect  seemed  to  be  corroborated  in  the  factor  analysis- 

In  addition  to  the  bipolar   scales,   students  were  asked 
to   rate  the  professors  on   an   overall  performance   scale. 
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Multiple  and  stepwise  regression  efforts  were  done  using  the 
overall  performance  evaluations  obtained  from  the  SOT  forms 
and  at  a  later  date  coincident  with  taking  the  computer 
survey.  The  icportant  information  gleaned  froo:  the  regres- 
sion analysis  suggested  that  the  six  bipolar  scales  used  in 
the  computer  survey  did  a  poor  job  in  explaining  students' 
perceptions  of  instuctor  overall  performance.  The  coeffi- 
cient of  multiple  determination  for  the  regression  model  was 
a  mere  .138.  Stepwise  regression  analysis  indicated  that 
the  scale  'applied  vs.  theoretical  course*  accounted  for 
most  of  the  variation  in  the  dependent  variable,  overall 
performance,  for  its  set  of  scales.  Thus,  it  would  seem 
that  whether  students  perceived  a  course  taught  by  a 
professor  as  being  applied  or  theoretical  had  a  more  signif- 
icant bearing  on  the  overall  performance  of  the  professor 
than  the  other  scales  in  that  set.  In  any  case,  none  of  the 
scales  proved  to  be  statistically  significant  at  the  .05 
level. 

The  stepwise  regression  performed  on  the  SCF  data 
yielded  a  coefficient  of  multiple  determination,  F.2  ,  of 
.986.  This  rather  high  value  of  R2  seemed  to  suggest  that 
the  scales  used  in  the  SOT  forms  more  nearly  explain  the 
variation  in  overall  performance  than  do  the  scales  ustd  in 
the  computer  survey.  'Course  organization'  accounted  for  74 
percent  of  the  explained  variation.  Even  with  this  high 
value  of  R2  ,  most  of  the  coefficients  of  the  independent 
variatles  turned  out  to  be  statistically  insignificant  at 
the  .05  level.  This  indicated  that  hiyh  multicoilinearity 
existed  among  the  scales.  A  check  for  multicoilinearity 
proved  positive  in  both  sets  of  scales.  Each  independent 
variable  was  regressed  over  the  other  independent  variables 
and  high  values  of  E^  resulted.  The  multicoilinearity 
problem  suggested  that  a  number  of  the  scales,  particularly 
on  the  SOF  forms,  be  combined  into  one  scale  or  measure. 
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In  addition  to  regression  analysis,  a  factor  analysis 
was  done  on  both  sets  of  scales  and  three  separate  factors 
were  obtained.  Factor  1  was  composed  of  the  three  scales, 
'grading  policy',  'effort  required  outside  class'  and  'pace 
of  course'.  Apparently,  the  judges  found  these  scales  to 
interact  consistently.  Factor  2  appeared  to  describe  a 
composite  effect  between  'class  size'  and  'course  relied 
upon  prerequisites'.  The  factor  loading  was  positive  on 
'class  size'  and  negative  on  'course  relied  upon  prerequi- 
sites'. One  might  infer  that  the  larger  the  class  size,  the 
less  that  course  was  perceived  as  requiring  prerequisite 
courses  in  Operations  Research.  This  appeared  to  be  true 
since  most  of  the  larger  classes  occurred  in  the  beginning 
of  the  curriculum.  Factor  3  seemed  to  describe  a  student- 
professor  interaction  effect.  The  correlations  among  the 
variables  in  this  set  were  high  contributing  to  high  factor 
loadings  on  nearly  all  variables. 

A  disjoint  cluster  analysis  was  conducted  or.  the  coordi- 
nates generated  from  the  multidimensional  scaling  algorithm. 
Each  professor  was  grouped  into  one  and  only  one  cluster. 
Exit  interviews  with  students  suggested  that  five  clusters 
appeared  to  be  an  appropriate  number  of  groups. 

B.   COMCIOSIONS  AND  BECCOMENDATIONS 

The  multidimensional  scaling  technique  seemed  to  provide 
a  weak  explanation  of  instructor  groupings.  The  reasons  for 
this  lEay  be  several.  First,  a  linear  causal  relationship 
may  not  be  appropriate  in  describing  students*  perceptions 
of  instructor  performance.  Certainly  the  six  bipolar  scales 
used  in  the  iotaractive  survey  were  not  powerful  explanatory 
variables.  However,  there  seem  to  be  one  or  two  strong 
indicators  among  the  scales  used  in  the  SOF  forics. 
Specifically,    a    student-instructor   interaction    effect    and   an 
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instructor  organizational  and  preparation  effect  appear 
dominant.  What  needs  to  be  done  in  the  fature  is  to  obtain 
further  scales  or  characteristics  from  Operations  Research 
students  possibly  during  exit  interviews.  Also,  one  aight 
suggest  that  another  look  be  given  to  changing  the  current 
SOF  form  as  it  stands  by  combining  some  of  the  highly  corre- 
lated variables  and  adding  new  characteristics  that  later 
prove  meaningful.  However,  it  should  be  noted  that  what 
might  be  considered  an  important  characteristic  in 
describing  instructor  performance  for  one  student  group  may 
prove  to  be  less  important  for  another  group. 
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APPENDII   A 
INPUT    VALUES    FOR    MOLTIDIMENSIONAL    SCALING    PROGRAM    KIST 
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22. 

16.     21. 

22. 

1  =  . 

16.. 

6. 

12. 

15. 

20. 

21. 

12. 

16.     11. 

23. 

13  . 

17. 

6. 

22>. 

16. 

21. 

22. 

23, 

22. 

16.     12. 

23. 

13. 

17. 

6  . 

23. 

16. 

22. 

22. 

23. 

22. 

22. 

2.       ^. 

-v. 

2. 

3. 

1. 

4. 

3. 

4. 

4. 

4. 

4. 

'*. 

5. 

i.       7. 

7. 

5  . 

1. 

A. 

7. 

5. 

5. 

6. 

7. 

7. 

6. 

6.       2. 

ccmpl;t£ 

STHP 
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APP2NDII    B 
FODB    DIHEHSIONAL    SPATIAL    MAPPIiiG    OF    PROFESSORS    POSITIONS 


5 

Z 

UJ 

Q 


1 

0 

90 

d 

- 

J 

o 

_ 

M 

A 
1 

1 

1 

- 

!                        1 

F 

1 

a 

1 

G 

-0.5 

0.5 

1.0 

L 

0 

C 

^4 

d 

1 

- 

H 

P 

K 

DIMENSION   1 
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N 

d 


z 
o 

2 

2 
O 


J— L. 


-0.5 


0.5 


1.0 


DIMENSION   1 
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lO 

d 


z 
o 

i7i 


d 


5 


-0.5 


O.l      r 


d 

I 


o 
I 


DIMENSION    1 
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»o 


c 

in 

z 


■V 


J ii I L 


0.4 


0.8 


1.2 


c        9 


DIMENSION  2 
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■z. 
g 

C/l 

z 

UJ 

Q 


1 

0 

1 

N 

C 

1 

to 
6 

d 

CM 

d 

- 

a 

A 

I 

I 

1 

1 

I 

p 

e 

H 

-0.4 

L 

d 
1 

- 

1 

0.4 
U 

J 

0.8 

1.2 
0 

DIMENSION  2 
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(O 

d 


z 
o 

z 

Ul 

2 

o 


-0.4 


d 


0.4 


d 

I 


o 
I 


OIMCNSION  3 


70 


APPEN5I?  C 
FIVE  DIHEaSIONAL  SPATIAL  MAPPING  OF  PEOFESSORS  POSITIONS 


z 
g 
to 

z 

LJ 

2 
Q 


0     ^ 

1 

so 

d 

- 

J 

d 

- 

f 

U 

A 
1                     1 

1                 1 

1 

B 

1 

-0.5 
C 

C 

0.5 

1.0 

L 

0 

E 

d 
1 

- 

H 

K 

DIMENSION  1 
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o 
(/I 

2: 

a 


0 

d 

- 

M 

N 

- 

E 

J 

1 

0 

- 

H 

F 

< 

1            A 

1 

1                   1 

1 

1 

G       0-0.5 

0.5 

1.0 

L 

C 

0 

d 

1 

d 

- 

P 

B 

DIMENSION   1 
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5 
(/I 

z 

C 


-0.5 


'0.5 


1.0 


o 


DIMENSION   1 
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.n 


g 
in 

z 

5 
Q 


J L 


-0.5 


0.5 


1.0    B 


o 
I 


DIMENSION  1 
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z 
g 

z 
u 
2 
5 


-0.4  0         G 


o 
C  I 


6  t- 


0.4 


0.8 


1.2 


DIMENSION  2 
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E  L  C 


g 

2 


-0.4 


0.4 


0.3 


1.2 


o 
I 


DIMENSION  2 
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in 

■7 

5 
00 

z 

Q 


H         E 


1^1  I  I  I 


-0.4 


0.4 


0.8 
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o 
I 


DIMENSION  2 
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0  c 


g 

z 

L.J 

2 
Q 


-cPa 


-0.4 


0.4 


0.8 


o 

I 


DIMENSION  3 
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D 

- 

d 

- 

I 

c 

IT) 

Z 

o 
in 

z. 

0 

1               • 

1 

-       F 

H 

E 

1 

M    , 

M 

2 
O 

-6:s 

B          -0.4 

L 

C 

-    < 

• 

0.4 

0.8 

■- 

6 

1 

J 

DIMENSION  3 
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in 

o 

z 
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-0.4        B 
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0.4 


o 
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DIMENSION  4 
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&PPENDII  D 
COaPDTER  SOORCE  CODE  FOa  I1JT2HACTI?E  COSPOTER  S0B7EY 


C  «                                        A     LITTLr    MOTf!     A8CUT    'HC     SURVEY 

C  * 

C  *  TH!;     =CLLGWING    CC'^PU"^  =  P     C10E    wAS    WRI'TPN    AND    I-PLc- 

C  *  W=MT:D    for     TH<=    PUOPT'"    GF    QE^arNING    PPOXL-^ITY    DATA, 

C  *  !.:.     SlMILiPITY/O'SS  IMTLAO  [■'Y    DATA    CN    jTJOF.IT    P5.'- 

C  *  CEPTtCN    OF     TFACHIi.G    ?E  ^FOk  .''A'lCc .        STUDF.'lTC    ARE    ALSO 

C  *  asks:     TC    rate    professors    on    ScV-RAL    eiPCLAR    SCAL:S. 

C  *  THJ     3LRVEY     IS    CO.'^POScD    PRIVJ^ILY    OF    FCjo    COMPCUaNTS. 

C  *  THESr     INCLUCF    THF    Cnr^TROLLING    EXECUTIVE    PROGRAM    •^A£,       • 

C  *  A    PASEL    OF     PP'^FHSJnRS    MAMiS    (SURVL     PAN'L)     CRcAT'O 

C  *  ENTIRELY    BY    THF    DISPLAY    MA.-'AGr -ENT    SYSTEM     (D'lS),     A 

C  *  FILE    TO    STACK    STU^^flMTS  '     RESPi^'lSCS     (SUPVIAS    DISPLAY) 

C  «  AMO     THE    ACTLAL    FC'TRAN    PROGRAM,     SUPV3    FCRTRa.N.       THE 

C  *  CHOICE    HF    FCR'RAN    AG    THE    LA^GUAGc     FOR    TH=     SJRVGY     >*&': 

C  *  OP.tV-.N    PRIMARILY    BY    Th:     PR  CCRA-'MERS  '     FAMILIARITY    WI^h 

*  TH"    LANGUAGE    AND     ITS    ABILITY    TO    SATISFACTORILY    HANOLF 

♦  THE     TASK. 


C  THE    EXECUTIVE    PPQGRAJ'    MAS 

STRACS    ERR 

*  acc:es£  display  management  system   (QMS)   M^cHri  = 

EXEC    OMS 

*  CALLS    PANEL    TF    PROFESS-T^S     (SURVl). 
«  DISPLAYS    ?"0F^S5DR     PAMFL. 

»  STACK    iELECTHD    PROFESSCRC. 

EUDEXSCZ    SURVAS 

«  PLACE    SELECTED    PRQFESSCRS    IN    TEMPORARY     CMS     FILE 

* 

XED  IT    OR    DATA     A  (NGFFCF 

*  START     INTERACTIVE    QL£STICNIMG. 

*  OUTPUT    RESULTS    TO    FILE    ^'OZFOCl 
FILEDEF    01    DISK  OR     OaTA    A 

«  A    MOCULE     (SURV3    MHOLLE)     WAS    CREA^^EO    TQ    ALLEVTATE    THE 

«  PROBLEM    OF    EACH    STUDENT    NOT    HAVING    THE    NECESSARY 

*  FORTRAN    CCMP^LEi^-        THF    PROGRAM     (SURV3    rQCTRAN)     IS 

*  LOADED    AMD    3EGUM    MORE    EFFICIENTLY    WI'H    THIS    DEVICE. 
« 

SU?(V3 

ERASE    OR    CATA 

*  FTNC  CUT  RrSPONOE.MT'S  US'RID. 
«  WITH  THE  IPF  FUNCTICN  USIRIO. 
« 

USESID 

*  PLACE  USEHID,  DATE,  S  TIME  IN 

*  EXcC2  VARIABLES. 
* 

CR-AO  VARS  CDUMMY  SUSERID  &DUMHY  COATc  &TI ME  CDAY 

» 

»  GIVE    QUTPtJT    RESULTS     FIL'MAME    OF 

*  THE     RE5PCN0ENT    AND     SEND    FIL2     TO 

*  3177P    WITH    DI  SK    DUMP. 

*  3177P    IS    THE     PRCGRAMfCRS'     CURRENT    ACCOUNT     *. 
* 

R91AME     FILE     Fr02F00l    A     SUSIIRID    PROFS    A 

*  "ACH    FILE     rS    GIVE.")     FTXEO    LOGICAL 
RECORD     LENGTH    OF    dO    CGLUMMS. 

CCPYFILE    SUSE^ID    PRCFS     A     C^ECFM    F    lRECL    30 

*  «4i    qp;^£f<g-q    jz    SPQCL     PUNCH    TQ    RESEARCHERS    USERID    » 
CP    SPOOL    PUNCH    TO    3177P 

*  FIL-S     ARE    SPOnLED    ANC    PUNCHED    TO    REScARCHER 
DISK    DUMP    iUS'RTD    PPCFS 

CP    SPQDL    PU*"ICH    CFF 

*  WHEN    SURVEY    IS     COMPLETED,     THF    FILi    OF    RESPONSES 

*  IS    FRASED    FROM    THE     RESPOflOENTS    A    DISK 
ERASE    CUSER  10    PRCF5 
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c 

c 

SURVAS- 

c 

c 

c 

c 

c 

c 

US=     PANEL 

SURVl 

£01    =    _ 

&C2    =    _ 

G03    =    _ 

£C4    =    _ 

&C5    =    _ 

&C6     =    _ 

C07    =    _ 

SC3    =    _ 

&09    =    _ 

SCIG    = 

&CH    = 

S012    = 

£C12    = 

5C14    = 

&315    = 

£316    = 

&C17    = 

&Cia    = 

• 

SCI";  = 

iC2  0    = 

&C21    = 

£C22    = 

&C2  3    == 

£02  4    = 

£D2;    = 

&C2  6    = 

&C2  7    = 

6C2  8    = 

&C2  9    = 

&C3C    = 

&C31    =■ 

., 

£03  2    = 

&C3  3    = 

£C34    = 

(:C3  5    = 

£036    = 

£C37    = 

£C3  3    = 

£C3<;    = 

DISPLAY 

£STACK 

I 

OL£OL 

££TACK 

T 

02  £2  2 

£5TACK 

03  £0  3 

£STACK 

0^£0'^ 

£STACK 

05  £0  5 

£STACK 

06£D6 

£STACK 

07£07 

£STACK 

08  £0  3 

£STACK 

09  £0  ; 

£STACK 

L0£01C 

£STACK 

li£Oi  1 

£  STACK 

i2£0L2 

£ST  AfK 

13£013 

£STACK 

14£0  14 

£STACi< 

15  £01  5 

£STACK 

16£0  16 

£STACK 

17  £0  1  7 

£57ACK 

ia£Oia 

£STACK 

19£019 

£STACK 

20£DZa 

£STACK 

21£Dii 

£STACK 

22£022 

£GTACK 

23£D^3 

CALLS    THf    PAfJcL    OF    ^ogFESoCPS     (SUP.Vl     PAN=L) 
AMC    STACKS    THr    RSSFOMS.tS.    THF    NUMBrR    IF    PL.NC- 
HOLDf:RS    JEPHMOS    UPTM    TH=     NUf'PER    TF    PROF  ES*^":.^: 
LISTcC    IM    THJ:    PAMEL.       IN    OUP    CASF     THERE    WE".S 
39. 
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&3.  Acx   r  2^^;dm 

iSTACK  I  2birj25 

S£TACK  I  26tl32t 

aSTiCK  I  27iD-?7 

dSTACK  I  28C~!£ 

&5TACK  I  2SclC2<5 

CSTACK    I   ior.niQ 

S5TACK  I  31;1D3  1 

SSTACK  I  32SD3  2 

iSTACK  I  33fl2J3 

aSTACK  I  34f:03A 

SrTACK  I  35fl0  3  5 

aSTACK  I  ibuDit 

JIGTACK  I  37r.0  3  7 

i.'^TACK  I  33  CO  3  a 

&5TACK  I  39tlD39 

&STACK  FIL" 

C  SUPVi    ?A.\^L 

c 

C  TH^     =CLLGwI^G     PSN^L    COriTiIN!A    LIST    CF    39    P:^3FFSC.jP.  S 

C 

c 

-.THr     FOLLOWING    3UPV;>    IS    0-::SIGM"C    TQ    CGLL'C*    OA"^A    ON    .'^TUO'-MTS' 
-PFRCEPi  r:  IS    "F     !MSTSUC"f"!^     £  rF  =  C'I  V'^:!C£  f  .       E.TE?     'HE     VALJ5     I 

-IN  THp  uNiEFScn'^sc  PC3r"^inr!  TT   Thc  >_.:ft  of  .-ach  ?rof"ss>:a  =''nM 

-.».HGM    YOU    ^AV■.     TAKi!.'!    AT    LEAST    ONc   CQUP.SF. 

_i-4LAN    '.MiHeUR.N  3-.-:.LVI"!    5NCSUS  «-00NAL3    SARR 

_i-.G-RAL!J     HPC'^N  .Ji-JAMtr     iAGLE  a-JAMES    cSA?Y 

_i-FAlJL    FI^Ct-tiECK  i-M"=AGL£    FCPRF""  S-JmCK    JAFFOPO 

i-OaNALD     jAV-R  _3-.JAMc5    HAPTMiN  ~5-.Th1mAS    HO  I  V  I  K 

.3-GILe'?i=T    HC'-mP-O  S-^AYM'=    HUGHnS  "S-GHARL^S     HU"^CHIM?: 

,      ---""-'.        ..^-,^^  .      ^     .     .„^^..  .^^^_^j,      ,_c^T3 

■«-PAJL    *^ILCh 

.w    w^^ — w--.     .-^.w  _,.  ..^....^     ,-,..,  i-HARCHMAfi     PEPRY 

S-.GARr     ^n:CK  3-GARY     PIRTTR  S-3G35RT    ^=A0 

S-SLSS    RI:hAPOC  "S-.P\T    3A:10C2  "a-aPJNQ    SHU3rR~ 

I^-SEX    ':Hy)CE  la-APMAriD'}     SCL-^S  'i-'lICHA.lL     SrVER=IGM 

i-JCESPH     STSWART  a-TIM    SULLIVAN  i-JAMES    TAYLCR 

_i-.K  =  vri     WGCC  _a-.JAMES    YEE  _S-.P=TE:^    iZHNA 

-CEPPESS     THF    INTFP    KEY    WH=N    FINIShFO 

C  SURV3    FORTRAN 

C 

C  VARIABLES     CE^INrC 

C 

C  I, J-    TH3S^     INTEGER    VARIABLES    AP  c    liS^U    X-,     SIMPLE 

C  THw     3RrG?^AM 

C  N       -    R^F-^S    TO    TH£    NU.""?'?    OF    PRCF'ESSGR;^    US'tH    3N 

C  C       -    RfcFi^.S    TG    ThF    T'-AL    'IUM9ER    TF    POCFESSCRS    K) 

C  IHG     TC    =ACH     riCIVIDUAL    RES  F  ^'ID^'IT 

C  CI     -    uQUAlES    TP    TH-    VALUc     C-l    FCP    t  ICH    RCSPOrJOEN 

C  LUll?     THE    I^ESOINOE'-T    ^HRU    Tl-E    PAIRWISE    COMPARISONS 

c       z     -  riT-3?R  cr:uNT~K  si-^ilaf  to   I  AMO  j. 

C  £(39  )-    DEFINES       THE    '^^SPOnnE'^TS     UMQU.-;    SUBSE'^    3F     ^RnFESS':■^S 

C  INOtERRCHM  -    V.aniABL.^S    US-D     IN    AN    rRf-CP     HA.'inLING    SUR^UTTJS 

C  "rRRSh'    POUND     IN    VS    =ORTPAN    MANUAL.        HELP'S     '0 

C  MAKE    ■^HE     SURVFY    USER    FRIENDLY     BY    CATCHI'IG     !N- 

C  CCPPFCT    R=S?0;JS:S,     i.e.    hitting    THt     ENT-R    KTY 

C  WHr.J     ^    'MUM':siC     '«Aj    .^^EQUT;  S  i  "  J. 

C  R(39,39)-    THF    TWP    OlMFMSnMAL    AhlRAY    OF    RESPCNScS    TO     SIMILAPITY 

C  3ATA     CQP    Th;-     RcSPQNOENT.       THE    R,-ASC.'J     I"     IS     D:mhMSI2N=D 

C  iT    39    IS    3ECAUS5    OF    THF    NUMBER    CF    PROFESSORS     IM    TH" 

C  P4NFL 

C  B0S{7,39  )-:  J  VILA  R    TC    APfV!     IXC'PT   THAT    IT    STORIES    THc    R'ES??NS"S    'D 

C  THE    BiPCLAR    "SCALES 


.S-GILei^cT    HC'-mRO  a-^AYM'=    HUGHES 

.i-.3ATR!Cn     J'^.COaS  .i-HAnPLD    LA''Sn,''J 

S-tiLINN    LINCSAY  i-AL.v,N    'ICMA'^Tl'^S 

.a-«C'aUGLi^S     NEIL  _a-"A;-MJFL     P4PRY 


COUNTERS 

through: 

U 

~H? 

THE 

P  4N"l 

suos: 

.     (SURVl) 
r     OERTAI 

N 

IT. 

USED 

TO    HF.LP 

83 


c 

C       L:ST(39)-     1     CHA^iCia?     dPP.4Y     .?"=    T  t- "    PSGF'^SSCPS     iAM'5 

C       A(39)-    i     C-iAR.VC'^P    aRFAY    THAT    STCP=S    ^Hf.    CHAR/^C'I^     i    ?.?.     A    BLAIiK 

C  OE?:NOirJo    UP'N    Hf.'W    THE    RHSFOMOS.NT    CO^.PL^TI^D    THE     PANEL    OF 

C  P=',C=-SSaRS 

C       Y-    A    CHARA:TF?.    VAPTASL':    that    CH:CKS    ^n    ^-^     if    THS    R'^SP'^NDF.i^iT     ANr-W=R=D 

C  CCRP=CTLY    ~u    CCNTLNU^i    QM     ^ITH    'H=    lURV'iY 

C 

TNTrG"=     T,,'nC,J.Z,Cl,8(39  )  ,  IM  C  ."IRRCOM 

PEAL    R(  J<;,39l  ,  EPS(7,39) 

CHARACT  •;R*20     LIST(39) 

ChARAr,-^:R>4    A(  39) 

CHAPACT1P*1     Y 

CC.MMCn/  :RRC2M/rND 

E;^T:!F,  ejAL     MYE.'^R 

CALL    cS'SET  (215,256,-1.,  I,  "'YSRR.O) 

C  RiVEMBES    HE    USED    39    PRQFESS-::RS    IN    QUP    PAIl^L    ISURVL) 


N  =  29 
C 

C  m'lALlZE    ARRAYS    P     AtiO     =PS    TQ     O.C 


DC    5    1  =  1, N 
CO    o     J=L,N 
R(!  ,  Jl  =C.O 
6  CCNT^NJE 

5  CCNTI.'.'U": 

00    a    I =L  ,7 

CO  9   j=  i.;i 
sp<:  ( ' ,  j)=o.o 

9  CCNTTMUH 

3  CONTIMUr 

C 

C  QzFl!\z    "^HE    CHAPACTEP    ARRiY    OF    PPCFES£ORS    LI3T 


C 


LIST(  l)  =  'V,i*SHBLRN,    ALAM    P.' 
LI3T(  2)  =  'A'IOr  U3,    ALVTM    =.  • 
LIST(  3  J  =  'e>'RR  ,     OHNALO    R-  ' 
LIST(  ^)  =•  BF.IWN  ,    Gr;;ALO    G.  • 
LI5T(  5)= 'EmGLE  ,     JAMCvM.' 
L  IST(  o  )='F£ARY  ,    JAfES    D.' 
LIST!  7)  =  «=ISCHerCK,     PAUL    :^ .  ' 
L!ST(  8)= 'FORRrST,     R.    N£AGLE' 
L  IST(9  »  = 'GAFPCO,     JACK* 
LISTl  10)  =''-.AVSP,     OC.NALD    P.' 
LIST(  li)  =  'HARTMA;j,     JAME.S     K,  • 
LrST( 12) ='HQI VIK ,    THOMAS     H.' 
LIST(  13) ='HOWAPO  ,    GILBERT    T. ' 
LI5;T(  1^)='HUGH5S  ,    WAYNE    P." 
LIST(  15)  ='HUTCHirvS,    CHARL'S    W.' 
LIST(  16)  = 'JACCeS,     P4~''.ICI\    A.' 
LIST( 17) ='LiR?CN,     hi«nLO    J.' 
LIST(  13)  ='L2WI£,     FE"^'-R     A.     W.  • 
LIST(  19)  =  'LIN0SAY,     GLEMN     F.' 
LIST( 20) ='MCM6STERS ,    ALAN    W.' 
LIST(  21)  =  "'1ILCH,     ^AUL    R.  ' 
LIST(  22)  =  '*;EIL  ,    DCUGLAS    E.' 
LIST( 23) =«PARRY,     iAfUHL    H.' 
LIST(  24)  =  'Fci- K>,     F.    MARCHMAM' 
LIST(25) ='°OnCK,     GARY    K.' 
LI£~(  26)  ='?0.'^TcR,    GA  =  Y    R.' 
L  IST(  27  )  =  'REAO,    RCBE^T    R.' 
LIST( 28) ='R!Ch4fiCS,     F.    RUSSFLL* 
LIST(  29)  ='':a:j:)CZ  ,    PA~?.!CK    A.' 
LIST(  30  1  =  ";HU86R~  ,     R^UNu    0.  • 
LISTC  31)  ='SHlJDCc  ,    PFX    H.  ' 
LIST(  32)  =  'SOL  I  S,     aP^TIDG    "  .  • 
LI3T(  33)  ='"^aV'^PE;GN,     '^rCH-^EL     G' 
LIST(  3^)  ='ST:^WART,    JO-'^IPH    8.' 
LI3T(  35  )  =  'SULLIVAN,     -I'^HTHY    J.' 
LI  ST(  36)  ='TAYLCR  ,    JA'^FS     G.' 


8U 


2C 


ia 

c 
c 
c 
c 
4a 

11 

12 

13 

30 

31 

32 

33 

34 

35 

36 

37 

41 
42 
43 

44 
45 
46 
47 

4g 

53 


EN'^ER    TN    VALUcS    ?CR     S  I.'-"  I  L  A  S  I"  Y    C?^ 


LIST(  37)  ='WiJCD  ,     -!.     K.  71  fJ' 
L I3T(  jd)  =  •Y  =  =  ,     JAMf S    ^.  • 
L  I5T(  39)  ='Z3HN^,     FFTr-:^    ^.' 

OBTAIN    TH'H    SUBSFT    OF    PPCFESSORS    SEEN    3Y    EACH    ^E".?ONDEUT 

C=l 

DC    LO     I=U''; 

BSAO  (  L  ,20)A(  T  ) 

Fa3M<^T(  2;<,Al  ) 

IF(A(  I  )  .EQ.'  I  '  )    THEN 
a(C  )=f 
C<+1 

ELSH 

SNO    IF 
CCNTIMU.: 
C=C-1 
C1=C-1 

EXPLAIN'    THP    RrC'J=ST     ^np     SI  >^  IL  AR  I' Y/0  I  3S  H'TL  AR  I  TY    OATf, 
F^CM    TH':    R  •;SFC:iO!!  :JT 

CALL    FRfCV*;     (  'CLRSCRN     '  ) 

WRIT  (6.11) 

FORMAT ( IX ) 

WRIT-^  ((5,12) 

FGRHAT(  LX) 

WRIT^ (6,12) 

FC'^AK  lA) 

WRIT- ( 6, 3C) 

FORMAT  (  IX,' Y:U    are    Nf:w    READY 
*OISST MILAR'TY     FOR') 

wRITi  (  tj,  2  1) 

FORMAT  {  IX, 'EVFPY     CCMBINATIOM    CF    TWO    PROFESSORS    ACCORO'MG    '3    YC'JR     = 
*f:RSOflAL    LIST.       ONC^') 

WRIT- 16, 22) 

FORMAT  dX,' AGAIN,    PL'^ASE     EMTEP.    IM 

«nN  OF    SI  f  :la:^i  ty  cf'  ) 

WRI-c(6, 22) 

FCR^^AT  (LX,'T':-ACHUjC    ^FFECTIVEriESS    USir;G    TH3    SCALE     PRQVTOiO. 
*    RESP  OMiES    TO    AT     ') 

WR IT? (6, 34) 

FORM^T(  1X,'.M0ST    rj\r-    DECIMAL    POINT. 
*ARITY  /DI SSIMILAR-  •) 

WRITO-  (6,35) 

FORMi  T(  LX,  ■  ITY     SCiLT    APPEARS     BELOW.       FOR    HXAMPL-,     IF    YOU     PiRCElV:; 
«PPOF-  .     3PC/<N    AND"  ) 

WRITE  (6,26) 

FGR.MAT(  LX  ,' WCCC    ~C    81;    VERY    SIMILAR     IN    ■^'ACHING    cF  FICT IVEM  ESS  ,    YOU 
fMIGHT     C40CrE    TC) 

WRIT? (6,37) 

FnRMA-(LX,'ASSIGr    TH^T    PAIR    A    SCORE    OF    2.2    TO    REFLilCT    A    HIGH    CEG^? 
*E    OF    31''  ILARI^Y.  •  ) 

WRITE (6,^1) 

FORMA -( IX) 

WRITE (6,42) 

FaRMaT(I2C,'  I -I + -f 1- -I A -I I  ') 

WRITE(6,-^3) 

FCPi«4T(  '■20,  '  1'  ,T25,'2'  ,T30,  '  3*  ,T35,'4'  ,T40,«5',T45,  '6'  ,T50,'7'  ,T55 
*,  '8' ,  T6J  ,  "^)'  ) 

WRITE( 6,^4) 

FCRMAT(  T20,' : •  ,T60,'  :'  ) 

WRITc  (  £, A5) 

FORMA T( :i8,  'VERY  •  ,T59, 'VERY'  ) 

WPIT^  ( 6,46) 

FORMA  T(  ri7,'SIfILA<",T5o,  'DISSIMILAR'  ) 

WPIT'^  (6,  A  7) 

FORMA  T(  1.X) 

WRTTE ( 6, 49) 

FORMAT (IX) 

WQITT^(6,53) 

FORMAT  (  no, 'TO    CONTINUE,     DEPRESS     THE     LETTER    R.') 


A    VALU' 


iEFL:cTrr;G  your 


FOR    YOUR    CaNV-MI3^: 


PEPCHPTI 
LIMIT 
THE     SIMIL 
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c 

c  THC  rf,aC1:\   iOVA.Nc;s   to   ccir:G  pai-wise  CD.MPARTGOrjs  df 

C  P^CFfiSiCSS 

c 

RFAO(  5,52)     y 
52  FCSM/\T(  \1) 

IF(Y.Nt.'R')     Th:N 

GC    "0    ^£ 
ELSF 
cNO    IF 
DO    50     1=1, CI 

Z=I-t-L 

CC  60  J=Z,C 
CALL  ^AIk(I  ,  J,?-  ,C,e,LIST) 

c 

C  THF  3UR9CU  !NE  PAIR  CHMTFOLS  LOOPING  ^N 0  ALL  PATRwiSC 

C  C3MPAR15CNS  FDR  SIHILA.'^ITY  DATA 

C 

6C     CCNrrNU? 

50     CCNn"4U: 

65  CALL  FRfCVS  ('CL^SCRM  •) 
WRIT"  (6,66) 

66  FCPMt  T(  !.;<) 
WRIT- (6,67) 

67  FORMAT(LX) 
WRITT  (6,  70) 
WPTTii  (6,80) 
WRIT-  (6, -50 
kdir--  (  o,  103) 
WRir^  (6,110) 
Wfi!"r=  (6,  120) 
WRITr(6,  130) 

THH    P  =  3PCND'^rjTS    .UE    BPIHFED    :M    THE    3IPQLAR    GCAL^": 

writ:  (6,  140) 

FOpM-*'(  LX,'WO     APT    MOW    INTr-F.ESTcD    IN    CDLLcCTING    AODITIT'.AL    I  NFCRMAT 
«rCN    OM     :TH£R'  ) 
FORMA T{ IX,' ACPEC-j    CF    YOUR    PRCFE330RS    AND    THE    COURSES    ~H5Y    TAUGH". 

*  THF     Bt  PCIAR*  ) 
FC^-^ar  dX,' SCALE3    USLD    FOR    THIS    PART    OF    THE    SURVEY    AR"    SIMIL'R    IN 

*OcSIG-l     rc    TH'^  '  ) 
FCRMftT  (1X,'SIMILARITY/0ISSI,^ILARITY    SCALE.       PLEASE    Lr-«!T    RESPONSES 

*  Tl    :i  T     "CST    ~h:>  ) 
FORMAT(i.)(,iOECIfaL    PLAC3    EXC^C"^    F-LR    TH=     FIRST    BIPOLAR    ICALE    WH7CH 

*RECU!RES    CNF    CF' ) 

FQRM^T(  lX,'FaLF     INTEGER    RISPO^ScS.        IF    YOU    HAV:    HAD    ON"'    PROF^S'OR 
♦MHRE    'H^NM 

FaRf'tT  (ix,'GNCE,    YCU?    RESPONSE    SHOULD    REFLECT    THE    LAST    CO  UR  S"    TA  UG 
*HT    TO     Y]L    BY*  ) 

FnR'n\T(  LX, 'THAT    PROFPSSHR.       LET    US    BEGIN.') 

WRIT)-  (6  ,  150) 

Fr.qM*  ~  (  IX) 

WRITF(6, 160  ) 

FORMAT( IX, ' PLEASu    TYPfc    TH^    LZTTIR    R    TO    CONTINUE.') 

READ(  5,  17C)     Y 

FCRMAT  (  J  1  ) 

IF(Y.Mi.'R')     TH^N 
GO   "^0    6  5 

ELS- 

bNO    IF 

CALL    FCTC.'^S     I  'CLPSCRN     •  ) 

DC    130     1  =  1, C 

WRITr  (  6,  171) 

171  FQRM(\T(  ix) 
WRIT-  (6  ,172) 

172  FORMA  "'i  IX) 
WRITE (6,  173 ) 

173  FORMAK  1X,'THIS    SCAL=    DHALS    WITH    THE    TliMEFRAMi     in     THE    CURRICULUM    W 
«HFM    YOU    TOOK     Tl-E  '  ) 

WRI'^'^  (6,174) 

174  FOR-^A  TdX, 'COUPS"    TAljr,HT     BY    TI-:    PROFESSOR    LISTED.        FOP    "HIS    SCALl 


c 
c 
c 

70 

8C 

90 

100 

110 

120 

130 

140 

150 

160 

170 
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*-itn--   ONLY  nf:P  •) 

WRIT^(  6,  Hi) 

175  FCn-^AK  LX,' TP    FOUa    VaUPS     (1,2,3    1«    ^).') 
WRI'^  ( 6,  176) 

176  FnRHAT(lX) 
WRIT-  (6  ,  177) 

177  FrR.'1.\T(  u) 
WRIT"-  (6,173) 

178  FGRM::T(  i;<) 
WRTT'=  (  6,  179) 

179  FCPMAT  (  1X,T20,  •  I h -i |  '  ) 

V.RIT^  (6,  181  ) 

181  FORMA  T(  lX,-20  ,  •  1*  ,T35,'  2*  ,T50  ,'3'  ,T65,'  A«  ) 
WRIT  (6,182) 

182  FGP'nT(  ix,-2u  ,  ':  '  ,T35,  •  :  '  ,T5a  ,•  :•  ,T65,'  :  •  ) 
WRIT" (6,  183) 

183  FORMAT( T12,^( 2X  .'CCUPSc    TAUGHT')) 
WRIT  (  6,  18^) 

la^         FCRMAT  (lX,Tl(j  , 'LST    C^    2MD  •  ,  T  3  1 ,  '  3F.9    OP.    ATH  •  ,  T  46,  •  5TH    OH     bTH'.Jbl,' 
*7TH   TR     dTH* ) 
WRI^^P  (  e,  185) 

185  FCf'MflT  (  r9,A- (3X  ,•  CLiRT  =  R'  )  ) 
WRIT"  (6,  186) 

186  FCR'''£.T(  IX  ) 
WRIT  (  6,  187) 

187  FORMcT  (  U) 

WftlTl    (6,190)L  rST(E(  I  )  ) 
19G  FaR-'iT(  LX.'SMTcR    YCLR    R:;SP0:-JS^    Fu7'  ,  lX,<i20,2X, '< ') 

161  cqnt::iu  = 

I  N0=0 

R!=•^Ol  5  ,2CC,S'IL;=1c2)     ?PS  (1  ,8(1)  ) 
2C0         FaRMAT(F3.2) 

IF(3PS  (1  ,5(  I)  )  .G'.A.O)     TH=M 

GC    TQ     162 
ELSE 
3NC    I  P 
IF(  I'lO.  !C.O)     r,c    TC    163 

162  WRIT=(5,ib4) 

16-4  FCR"1AT(1X,  '  IMCCPRFC"    IMPUT,     PLHAS2    SNTHF.    AM     IMTCr.F?.    VLU:"', 

*  •     FP3M    1     TQ     ^. , ) 

RPWIIIO    5 
RFwrMO    5 

GO  rn    leL 

163  CCNTINU" 

CALL    FRTCVS     (  'CL.^SCRN    •  ) 
180  CCNT-^UU^ 

DC    210     1=1, C 
writ:   (6,  ISl) 

191  FORMkTIIX) 
Wfil''!  (6,192) 

192  FnRMt'(lX) 
WPITc{6,  193) 

193  FQRMATC  IX,' THE    FGLLCViinG    SCALF    C2iLS    WITH    CLASS    SIZE.') 
WRITT   (6,  194) 

194  FCRMATdX) 
WRITc  (  6,  195) 

195  FOR^«AT(  IX) 
WRIT^  (6,  196) 

196  FOR.^uTl  T2C,  •  I -»■ f + >■ ♦ h ^ I  '  ) 

WRITF (6,  197) 

197  FORMA T( T20,'1',T25,'2',T30,'3«,T35,'4',T40,'5',T4  5,'6',T5  0,'7',T55 
*,  '8',  ToJ,  '9' ) 

WRIT (6,193) 

1  C8         FCRMA  T(r25,':',T55,':') 

WRITS ( 6,  199) 
199  FCR.-IAT  (  r21,  '  VCPY     SVALL  •  ,T  5  1  ,  '  V'R  Y    LARGE') 

writ:-  (  6,2C1) 

2  CI  FORMAT  (  ''21,  'C  LA?  5    S  I ZE  '  ,  T  5 1 ,  '  CLA  ^S    SIZE') 

WRIT  (6,2G2) 
2C2         FCRMaT( IX) 

WRITF (6,203) 
2C3         FORMAT!  IX) 
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WRIJr  (6,220ILI£T(e(I ) ) 

220  FQRMa-{  LX,'£.'iTER     1(J0P    RESPCUS=.    FCS  '  ,  1  X  ,  A20  ,2X  ,  •  < ■) 

23i         CCNTT'IU: 

tN0=O 

REAO(  5,2  3C,ENn=23  2)     3PS(2fBl  I  )  ) 
220         FCP'^ftKsa.Z) 

IF(BP  2(2  ,?(!))  .G'. 9. C)     "'H'-N 
3PS( 2  ,i  m  J  =  BF£  (2,S(I)  )/  10. 0 

2ND    I  F 

IF(  P'O.IG.O  )    GC    TC    233 
232  ViRTT-(  3  ,23^) 

23^  FORMAT  I  !;<,' !MCCPP':CT    INPUT,     PL5AGZ    SMT=R    AM     iriT;r.:R    VALU  =  ', 

*  •     FPCM    1    ^C    9    CP     i    RF,AL'  ) 

URI-:{ S  ,238) 
238  FOR'tATC  IX,  'VALUE    FRCM     l.J    TQ    S.O.M 

RF'-^T  NO    5 
RSWT   JO     5 
GO   'D    III 
CCNTTMUi 
CALL    FRTCMS     I 'CL'SCRN    • ) 

DO    2^J     I  =l,C 
WRI-r-  (6,  211) 
FCF-«aT  (IX  ) 
WRTT^  (  6,212) 
FOR.-laT(  ix) 
WPIT^ (6,213) 

FCPMATt  iX,  '  TH?     FTLLGVJIMG     SCAL=    Cli/.LS     WITH    ^HcTH'R     "^HfE    CDUPSc    '«AS    'I 
♦PRE    THFIPETICAL    DP'  ) 
W  P  I  T=  { 6  , 2  1^  ) 

FQR.MaT(IX,'^P?LI':n    IV.    MATURE.') 
WPIT- ( 6,215) 
FORMAT  (1.x) 
WRI-^  (6,216) 
FCP.''4T(LX  ) 
»R!T- ( 6,217) 

FORMAT  (  r2C,  '  I H 1- -I *■ H + -t- I  •  ) 

WPITZ (6,218) 

FORf'laTC  T2C,  '  1  '  ,T25,'2'  ,T3C,'3  •  ,"^35  ,1  ^i  ,T40  ,'5'  ,T^5,'6'  ,750 ,  '  7  '  ,  "^  55 
*,  '3',T6J,  '<5'  ) 
WRITH ( 6  ,219) 
FuRMrtT ( r25, • :  •  ,T55,'  :') 
WRITr (6 ,221 ) 

FnRMA-^(  r22,'  4°PLi£D'  ,750,  'THE  CPFTTCAL  '  ) 
WRI~  (6,222) 

FCRMA-^  (  r22,'  CCUR£E'  ,7  5  3,  •  COURSE'  ) 
WRIT-  (6,223) 
FCRHA  T(  IX) 
WRITE (6,22^) 
FCRM4T(1X) 
WRIT= (o,250)L  IST(a(I  )  ) 

FQRMiTi  lX,'=NT;ER     YCU.^    RrSPONSS    FCR',  1X,A20,2X,'< ') 

COrUT'lU: 
INO=0 

REAO(  5,Z6C,EN0=252)     9PS(3,a(I)) 
260  FCRMft'(=3.2) 

IF(3PG  (3  tE(I  )  )  .G".9.0  )    -^MFM 

apii  3, 3(1)  )='iF:  (3,3(1)  )/ia.o 

ELSE 

END    :  F 

IF(  I;JD.  =C.O)     GC    TC    253 

252  WRn  =  (6,254) 

25<»  FaR;MAT  (  IX,  '  INCQ.'RECT     ^^!P'JT,     PLEAS!!    ENTER    AN     INTEGER    VALUE', 

*  •     FRIM    1    TG    9    CR    A    REAL'  ) 

WRT'E(6,258) 

253  FORM AT(  IX, • VALU"    FROM    1.0    TO    9.0.') 
REwPlO     5 

REWIND    5 
GO    TO    251 
2  53         CCNTiMU! 

CALL    FPrCVS     ( • CLR3CRN    '  ) 


233 

210 

211 

212 

213 

214 

2  15 

216 

217 

213 

219 

221 

222 

223 

2  24 

2  50 
251 
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2A0 


26L 

2  62 

263 

264 

265 

2  66 

2  67 

2  63 

269 

27i 

2  72 

2  73 

2  74 

2ao 

2^4 

290 


2  95 
2  97 


298 


2  96 

2  70 

28L 

2  82 

2  9L 

2  92 

2  93 

2  83 

2  84 

CCNTT  'JUl 

DO    270     •.=1,C 

WRITP (6,261 ) 

FORMft  T(  L;<) 

WR!"!-?  (  5,262) 

F  C  P  ."^4  r  (  L  X  ) 

WRtr  (6,26i) 

FOR'^t  n  U,  'THIS 
*0    THF     P^CF':SSOP 

WRIT- (6,26^) 

FQR,HAT(  iX,  'BE 

WRTT":  (6,265) 

FCRf^A  T(  IX) 

WRIT=( 6,266  ) 

FaR■^4T(  LXI 

WSnr  (6,267) 

FCR^AK  rzQ,  <  1 

WRirr  (  6,26d) 

FORMa'(  r20,  •  L  •  ,T£ 
* ,  '8'  ,T6)  ,'9'  ) 

WRIT"  ( 6, i69l 
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TC 


aSFLECT;    HOW    OIFFTCJLT    A    GR-.TFR     ("111    ???C"IV: 


) 


2'  ,T30,  '3  '  ,T35,'  't*  ,^40,'  5 


%5 


"5  0  ,  •  7  ' 


•55 


FORMAT ('25, ' :  '  ,T55,  •  : ' ) 

WPITt  (6,271) 

FORMAT  (  7  23,  ':aSY*  ,T54,'HA:^>0'  ) 

WRITF  (  6,272 ) 

FCR:-1AT(  T22,  'G3flO:P«  ,T53,''1RA0:?'  ) 

WRTTr(6,273) 

FrsMiT  ^l^ ) 

WRIT-  (  6,27^) 

FORMr.  T(  U) 

writ::  (6,230)L  I5T  (6(1  )  ) 

FOR.'^AT(  IX  .'SNTE.'.    >CL.'    RESPONSE    FC"  ,  IX  ,  A20  ,2  X  ,' < •) 

CCNTI.'JU-i 

INC=0 

RiAO( 5,29C,EMO=295)     8PS14,8(:)) 

FORMS  ~(  F3.2  ) 

IFCBP  S  (  t  ,a(  I)  )  .G'.9.0)    TH'^M 

BPS('»f)(I))=fiPS(4, 8(1))/ 10.0 
ELSE 
END    I  F 
I  F  (  IN  0  . 

wflrn  ( 


C    296 


:PPC' 
CR    . 


(  •  CLPSCRf;    '  ) 


■Q.J)     GC 
,297) 

FOR'-^ATC  IX,'  INCO 
*  •    F  •; : ,"    1    "^  Q    9 

WRIT'=  (t  ,298) 

FQR'^^Tl  IX,  '  VALUl 

REWTMO     5 

REWIND    5 

GC    TO     294 
CnNTIMU= 
CALL     FRTCMS 
CCNTT  flU-: 
DC    30  0     I  =  1 ,  C 
WRIT^  (6,231) 
FQRM4"^(  iX) 
WRTTl (6,282) 
FORMA -(  IX) 
WRITE (6,291  ) 
FCRMAT(LX,'THIS    SCAL 
*GHT.'  ) 
WRITE (6,292) 
FORMA T( IX) 
WRIT"  (6 , 293) 
FCR'"^T  (  IX) 
WRI-^:  (6,263) 

FORM^  T ( T  2C, •  I H 

WRITc  (6  ,284) 


lapuT, 

REAL' ) 


=  LEiS3    FMT'F.    ifj     IMTEGcR    V'.LJ^' 


FROM    1. 0    TO    9.0.' ) 


M=«!^UR£S    TH-    PACr    AT    WHICH    TrtF    CLASS    W^  S    T^U 


:ORMi  T(r2C,'l'  ,T25,'Z'  ,T30f'3 
^63,  '9'  ) 


■r 


,T35 


%0,'  5 


■5, '6' 


'50, '7' 


WRITF  (  6,285) 
FnRM6,T(  125,  • 
WRITE (6,286) 


,T55, 


89 


2  86  FC.'^''1AT(  T  23  t  •  :LCW'  ,T54,'  -AST*  ) 

WRI-r-r  (  6,287) 
2  87  FORMA  r(r22,  'PACE', TS'^, 'PACE') 

WBIT;^  (6  ,238) 

2  88       fcsmakl;*) 

WRITC( 6,289) 

2  89  FOB'-IAK  !.;<) 

WRIT- (6,310)LISTIS(I ) ) 

3  10  FCP'^ATdXt'EMTcR    YCUP.    RESPONSE    FOR  '  ,  IX,  A20  ,2  X  ,  '  < •) 

321  CONTTMU: 
INO=0 

REAO(  5, 3 20, END  =  3 22)     9?S (5 ,a(  I  )  ) 
3  20         FaRftf.T(:3.2) 

IF(RPS  (5  ,e(  I  )  )  .G^.9.0)    THFM 

aPS(5  ,  3i  I)  )  =  eFS(5,e(  I)  )/L0.0 
PLS'.= 
ENC    IF 
IF(  inO.=:C.3)     GC    TC    323 

322  WRIT:^{  i  ,32'^) 

324  FOR-^ATl  IX,  '  liiCCPRECT    INPUT,     PLEASt    EMTEF.    AN     INTEGER     VALUE', 

*  '     FR  1^    1    TG    9    CP     '.    RFAL'  I 

V.RIT=(6,328) 

323  FCR'^ATl  IX, 'VALUE    FRCM    1.0    TO    9.0.') 
REWIiJD    5 

REWIND     5 

GC   TQ     J21 
CONTTMUt 

CALL     FRTCMS     (  •CLRSCR^.     '  ) 
CCNT7  NUt 
DC    33  0    I=1,C 
WRITc(6,301) 
FORMAT  (  LX) 
•wRITF  (6,  i02) 
FCRV^TllX) 
WRIT=  (  6,2C3) 

FORMAK  IX, 'THE     p^jLLCUIMG    SCALE    MfiiURES    THE    AMGUNT    CF    EFF-;RT    REQUT 
*RED    dY     rhP    STUCHNT' ) 
WRIT^  (6,315) 

FORf-aTlLX, 'OUTSIDE    QF    CLASS.') 
WRIT^(6,3C4) 
FORMAT( LX) 
WRITE (b,305) 
FORMA  T(  IX ) 
WRIT= (6, 206) 

FORMAT  (  :20,  •  I ■*■ -" ■< H 1 ■> ->• I  ' ) 

writ:  (  6,3C7) 

FORMAT  (  r2Q,'L',T25,'Z',T3Q,'3',T35,''^'  ,TAO,'  5  '  ,'^'^5  ,  '  6 '  ,T5  0  , '  7  '  ,T55 
»,  '3' ,T63  ,'9'  ) 
WRIT-  (6,3CB) 
FORMAT! r25, ' :  '  ,T55, ' : • ) 
WRir=  (6,309) 

FCRMAT(  ri9, 'LITTL.:    EFF^PT  •  ,T50  , '  MUCH    EFFORT') 
WRI--(6, 311  ) 

FORMAT( ri7,' RECUIPED    OUTS  IDE'  ,T43, ' REQU IRcD    OUTSID£') 
WRIT^ (6, 2  16) 

FORMAT (723, 'CLASS'  ,T53, 'CLASS  '  ) 
WRIT--  (  6,212) 
FORMA T( IX) 
WPTT=(6,313) 
FGBM£.T(  iX) 
WRIT-^:  (6,  3-^0  )LIST(e(I  )  ) 

FOR.'lATC  LX,' ENTER    YCUR    R^SPQfJSC    FOR  ' ,  IX,  A20  ,2X,  •  < •) 

CONTINUE 
INC=0 

RFAOl  5,  350,HMD=352)     3PS(6,a(I)) 
3  50         FCRMtT(-2-2) 

IF(3PS(  0  ,e(  I)  )  .GT.9.Q)    th  =N 

BPS(6,J(I)  )=6P5  (6,BII)  )/  10.0 
ELSE 
END    I  F 

IF(  INO.'C-O)     GC    TC    353 
352  WRI'  =  (6,25't) 


323 

3  CO 

3  CI 

302 

3  03 

315 

304 

3C5 

3C6 

3C7 

3C3 

3C9 

311 

316 

3L2 

313 

3^0 
351 

90 


FTR-'a-t  IX.  '  IMCn-'nEC"'    IMPIIT,     OLEA'-.E    E'ITSR    AM     I,'|T£G''     VtLJP', 

*  •     FRjy    1     TC    9     C;      ,    RrAL'  ) 
WRITS (6  ,353) 

FCR"'AT(  LX,  '  VALU-    FRC  ^     L.O    TO     S.O.'I 
REWf  MO     5 
RCWriD     5 
GC    -n     J51 

CQNTTMU: 

CALL     CR  res     (  'CL.^SCRM     '  ) 

CONTT  MU: 

DO    360     1=1, C 

WRITE (6,33L) 

FaR;'1AT(  1.x  ) 

WRIT^  (6, 332) 

FCRMAT  (IX  ) 

writ:  (  6,323) 

FGRMftT  (LX,'THE    LAST    :^CALE    ASKS    TO    kHAT    EXTE'JT    TH2    Cruj^Zf     RELI5C    UP 

♦  ON    THR     2RFRFQU  I3IT5S.' ) 
WRIT: ( 6,335) 

FaRM£T(   t;(  ) 

WRIT- (6,336) 

FCRMaT(  IX) 

WRITE(6,337)  

FCP:-«A  r  (  'rZO,  '  l  •  ,TZ5,  '2'  ,T30,  '3  •  ,T3  5,'  <♦•  ,T^0,  -5  •  ."^S,  '6'  ."SO,'?'  ,T55 
«,  'as  ^63  ,  '9'  ) 
WRI-^'-  (6,  339) 
FCR.'-'AT  (725, •  :  '  ,T55,  •  :  •  ) 
WRIT- ( 6,3AI) 

FnRMf^(  T  L9, 'CCUP'E    RtiLI  ED  •  ,  T-V  9  , 'C 'JURSE    RELIED') 
WRITF  (6,  3^2) 

FQR'^aT(  121, 'L.TTTLF    C,N'  ,T5  1 ,  '  HEAVI  LY    QM'  ) 
WRITE (6,343) 

FOR-^ATi  T19,  '  =Rf=RiGL!  SITES'  ,T4g  ,  •  PPERSGU  IS  ITES  '  ) 
W  R  I  TF  (  t ,  3  -^A  ) 
FCRMAT  (IX  ) 
WRir=  (  6,  3A5) 
FGRMAT( IX) 
WRITE  (6,370  )LIS"T(e(  I  )  ), 

FCRMAT(  lX,'fHT*R     YOLrt    RESPONSE    =0?  '  ,  iX  ,  A20  ,2  X  ,  •  < ') 

CONTINUE 
I  N0=0 

R'=AO(  5,  3aa,£MD  =  332)     9PS(7,a(I)) 
380  FORM^T (-3-2 ) 

IF(3?'W  ?  ,e(  I)  )  .GT.9.0  )    TH-M 

BPS(7,a(i))=aFS(7,a(i) )/io.o 

ELSE 
END    I  F 

IF(  TMD.:G.O)     GC    TC    383 
382  WRrE(i,334) 

3  64  FORM\T(  ix,  • 'MCCBRCCT    INPUT,     FLEASi    ENTER    AN     INTEGER    VAL  J=  '  , 

*  •     FR  IH     1    TG    9    HR    i    REAL'  ) 

WRITE  ("j  ,388) 
388  FOR-IATl  IX, 'VALUE    FRCM    1.0    TO    9.0.') 

REWIND    5 
REMIND    5 
GO    Tl     3  81 
3  83  CONTirJUE 

CALL    PRTCMS     I 'CLPSCRN    • ) 
360  CONTINUE 

CALL     PRrC;''^     (  •  CLPSCr^M    •  ) 
WRITE  (  6,386) 
3  86  FCRr'tT(  IX  ) 

WRITE (6, 387) 
3  87  FORMA T(  IX) 

C 
C  THE    DATA     IS    COLLECTlD    AflC    STORED     IN     EACH    ARRAY 

C 

00    40  0     I  =  l,N 
WRITc(  2  ,410)  (PC  ,J)  ,J=1,N) 

410  FaRMAT(  2GF4.  1  ) 


3  54 

3  53 

3  53 

3  30 

331 

3  32 

3  33 

335 

336 

337 

3  33 

3  39 

341 

3  42 

3  43 

344 

345 

3  70 
381 
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4CJ  CONT''   JU' 

WfilTY  (2,415) 
4  15  fOTi^t-r  {  ^^^^ 

DC    42  0    r=l,7 

wRiT:-(i  ,430  )  (ep", (i,j)  ,j=i,;j) 

420  PGS''A"'(  2GF4.  1  ) 

420  CC.NTIHUl 

STCP 
FNO 
C 

C  SiaUFSTS    FDR    SIMILARITY /CISSI.MIL-iRITY    SCORES    AR€    MAD^    H^'^H 

C 

SL'S'^OUTINF    PAIR  ('  ,J,P  ,C,8,LIST) 
COMH-IM/tfSRCJM/INC 
INT'G.:R     !  ,C  ,B  (3<3)  ,J,K,L 
REAL    5 ( 39,39) 
Ch4f(fiCTTR*20     LIS~(39) 
CALL    FRfCN'S     (  'CL^SCRN    •  ) 
WRIT-  (o,  IC) 

WRIT---  (6,  11)     L!ST(e(I)) 
IC  FORMt\T(  r2  ,' OL-AS'-    ^n~^R     IM    YOUP    VALUE    FGR    SIMILARITY     Vi    rnACH^riG    = 

*FFECT  iVihEiS    CC.-^PARI^G'  ) 
11  FCRM(<,T(  IX, ;.20,i;<,' AGAINST    THE    FPOFISSOF     INDICATED    bY    TK-    ARROVl.') 

IF(J.LT.C)    THEN 
K=J+L 
WRr""=  (S  ,25)LTST(S(  J)  ) 

25  FnR'-1M(  2CX,  ;2C,2X,'< ') 

CC    30    L=K,C 
URI  ^■•Ae,j>5)  LIST(5  (D) 

3  5  FOpmaT  (2QX,A2a) 
3C              CC-JT  1,'JJE 

FLSE 
K  =  J 
WRIT  =  (3  ,40  )LIST(fl(K)  ) 

4G  cORM-M(2CX,i2C,2X,'< •) 

FNC    !  F 
HRIT^ (6,42) 

42  FORMfiT(  r2Q,'  I + ^ > + ♦ -t ■>- 1  •) 

WRir"  (  6,42) 

4  3  FCRMAT  (  r20,'  1'  ,T25,,'2'  ,T30,  '  3  '  ,T  35  ,  •  4*  ,T40,'5'  ,  T45  ,  •  6  '  ,  T50  , '7'  ,T  55 

*, '3', T6J  ,  '9*  ) 
WRIT?  (  6,44) 

44  FCP'''aT{  r20,'  :  '  ,T6G,  '  :  '  ) 

5  5  FORMA  ~(  'rie,  'VFRY*  ,T=S,  'VERY'  ) 

WRir=  (6,46) 
46  FOR^ftK  ri7,  •SI^'ILAP»  ,T56,  'DISSIMILAR'  ) 

50  CQNTINU: 

INO=0 

R£AO(  5,  •i5,5MD  =  61)    3(3(1  )  ,  8(  J)  ) 

45  FCPMAT (=3.2) 

IF(R( Q( I  )  ,a( J)  ).GT.9.0)     THCN 

P(B(i),e(j))  =  Ri3(:),a(j))/io.o 

ELSE 

END    IF. 

IFdMD.ZC-O)    GC    "^C    70 

61  WPI-^H(b,62) 

62  FCPM^T(  IX,  '  IMCQRFFCT    INPUT,     PLEASE    cNTF.R    A.'J     IMTEG.'P.    ViLJ"', 
=»  '     =R  ]H    1    TC    9    CR     'i    RFAL'  ) 

WRIT2(3  ,63) 

63  FCRMAK  IX,  '  VALUE    FF.LM    1.0    TQ    9.0.' J 
R='^I,'1D     5 

REWltO    5 
GC    ^n     5C 
70  CnNTI.NUr 

RFTlJPri 
END 
C 

C  FIPCR    HANCLING    SUBROUTINE    USFO     TQ    HELP    WF.  =  D    OUT     INCORRECT 

C  RESPONSES 


92 


SUPCCUT  tNE    MY=PP(  IF,; 
INT;G-R     IBt-TC",IERR 
CHAPtC~^'»L    CH" 
CC."'-"^f|/  f  POCG"^/  INC 
I  P  =  TC  .")  =  ] 
IMO  =  L 
PFTUrMJ 
«^N0 


:C3,I=3P,ChR) 
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